Jul 18 01:46:12.356273 ip-10-0-128-48 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 18 01:46:12.356287 ip-10-0-128-48 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 18 01:46:12.356295 ip-10-0-128-48 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 18 01:46:12.356742 ip-10-0-128-48 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 18 01:46:23.426612 ip-10-0-128-48 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 18 01:46:23.426628 ip-10-0-128-48 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 1c003390817145eab2e87207695d8318 --
Jul 18 01:48:14.444759 ip-10-0-128-48 systemd[1]: Starting Kubernetes Kubelet...
Jul 18 01:48:14.840183 ip-10-0-128-48 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:48:14.840183 ip-10-0-128-48 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 18 01:48:14.840183 ip-10-0-128-48 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:48:14.840183 ip-10-0-128-48 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 18 01:48:14.840183 ip-10-0-128-48 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:48:14.840956 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.840167    2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 18 01:48:14.846221 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846196    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:14.846221 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846218    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:14.846221 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846223    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:14.846221 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846227    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846232    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846236    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846241    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846245    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846256    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846260    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846264    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846268    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846273    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846278    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846282    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846286    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846290    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846294    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846298    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846301    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846305    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846310    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846315    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:14.846510 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846318    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846322    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846326    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846330    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846334    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846338    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846342    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846346    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846350    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846354    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846358    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846362    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846367    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846370    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846377    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846381    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846386    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846390    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846395    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846400    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:14.847369 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846404    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846408    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846412    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846437    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846445    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846450    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846455    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846459    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846463    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846467    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846471    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846475    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846480    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846484    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846488    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846493    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846497    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846501    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846505    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846509    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:14.848218 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846513    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846517    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846521    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846525    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846529    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846533    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846538    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846543    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846547    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846551    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846555    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846568    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846573    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846577    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846582    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846586    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846590    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846594    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846598    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846605    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:14.849036 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846609    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846614    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846621    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846627    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846632    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846637    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846641    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846646    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846652    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846657    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846662    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846667    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846672    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846677    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846681    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.846686    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847893    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847904    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847910    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:14.849906 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847915    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847922    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847928    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847932    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847936    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847942    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847949    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847955    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847959    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847963    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847968    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847972    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847977    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847981    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847986    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847990    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847994    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.847998    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848003    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848008    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:14.850643 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848012    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848017    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848021    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848027    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848031    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848035    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848039    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848043    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848048    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848052    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848057    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848061    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848065    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848070    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848074    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848078    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848083    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848087    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848091    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848096    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:14.851295 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848100    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848104    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848108    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848112    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848116    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848120    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848124    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848128    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848132    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848136    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848140    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848144    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848149    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848153    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848157    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848161    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848165    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848169    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848173    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:14.851852 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848180    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848185    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848190    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848194    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848200    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848205    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848210    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848214    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848218    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848222    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848228    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848232    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848237    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848242    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848246    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848250    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848254    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848258    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848262    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848266    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:14.852314 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848271    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848275    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848279    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848284    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848289    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848293    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848297    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848301    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848306    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848310    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848315    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848319    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848323    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848327    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848331    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848335    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848340    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848981    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848989    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:14.852843 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848994    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.848998    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849003    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849008    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849012    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849017    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849021    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849026    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849030    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849034    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849038    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849042    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849046    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849050    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849054    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849058    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849063    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849067    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849071    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849076    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:14.853303 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849080    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849084    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849088    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849093    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849097    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849101    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849105    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849110    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849114    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849119    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849124    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849128    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849136    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849142    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849147    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849151    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849156    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849160    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849165    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:14.853796 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849169    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849173    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849177    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849181    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849185    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849189    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849194    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849198    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849202    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849206    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849210    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849214    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849219    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849223    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849228    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849232    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849236    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849240    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849244    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849248    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:14.854277 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849252    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849256    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849260    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849264    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849268    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849273    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849277    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849281    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849285    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849289    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849294    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849298    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849303    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849306    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849310    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849315    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849321    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849328    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849332    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849336    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:14.854905 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849340    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849346    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849351    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849356    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849360    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849365    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849370    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849374    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849379    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849384    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849388    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849392    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849397    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849401    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849406    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849410    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849414    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.849441    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849547    2552 flags.go:64] FLAG: --address="0.0.0.0"
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849558    2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 18 01:48:14.855411 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849568    2552 flags.go:64] FLAG: --anonymous-auth="true"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849574    2552 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849581    2552 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849586    2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849592    2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849600    2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849605    2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849610    2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849615    2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849621    2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849626    2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849631    2552 flags.go:64] FLAG: --cgroup-root=""
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849635    2552 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849640    2552 flags.go:64] FLAG: --client-ca-file=""
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849644    2552 flags.go:64] FLAG: --cloud-provider="external"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849649    2552 flags.go:64] FLAG: --cluster-dns="[]"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849659    2552 flags.go:64] FLAG: --cluster-domain=""
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849663    2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849669    2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849674    2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849679    2552 flags.go:64] FLAG: --container-log-max-files="5"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849685    2552 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849689    2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 18 01:48:14.855928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849694    2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849699    2552 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849705    2552 flags.go:64] FLAG: --contention-profiling="false"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849709    2552 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849714    2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849719    2552 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849723    2552 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849730    2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849735    2552 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849739    2552 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849744    2552 flags.go:64] FLAG: --enable-load-reader="false"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849748    2552 flags.go:64] FLAG: --enable-server="true"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849754    2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849762    2552 flags.go:64] FLAG: --event-burst="100"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849766    2552 flags.go:64] FLAG: --event-qps="50"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849772    2552 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849777    2552 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849781    2552 flags.go:64] FLAG: --eviction-hard=""
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849787    2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849792    2552 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849797    2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849801    2552 flags.go:64] FLAG: --eviction-soft=""
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849806    2552 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849810    2552 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849815    2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 18 01:48:14.856488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849821    2552 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849827    2552 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849832    2552 flags.go:64] FLAG: --fail-swap-on="true"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849837    2552 flags.go:64] FLAG: --feature-gates=""
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849843    2552 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849847    2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849852    2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849857    2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849863    2552 flags.go:64] FLAG: --healthz-port="10248"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849868    2552 flags.go:64] FLAG: --help="false"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849872    2552 flags.go:64] FLAG: --hostname-override="ip-10-0-128-48.ec2.internal"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849877    2552 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849882    2552 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849886    2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849892    2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849897    2552 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849902    2552 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849906    2552 flags.go:64] FLAG: --image-service-endpoint=""
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849910    2552 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849914    2552 flags.go:64] FLAG: --kube-api-burst="100"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849919    2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849923    2552 flags.go:64] FLAG: --kube-api-qps="50"
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849928    2552 flags.go:64] FLAG: --kube-reserved=""
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849933    2552 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 18 01:48:14.857096 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849939    2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849944    2552 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849948    2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849953    2552 flags.go:64] FLAG: --lock-file=""
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849957    2552 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849962    2552 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849967    2552 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849975    2552 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849981    2552 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849986    2552 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849991    2552 flags.go:64] FLAG: --logging-format="text"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.849996    2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850001    2552 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850006    2552 flags.go:64] FLAG: --manifest-url=""
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850010    2552 flags.go:64] FLAG: --manifest-url-header=""
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850017    2552 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850123    2552 flags.go:64] FLAG: --max-open-files="1000000"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850135    2552 flags.go:64] FLAG: --max-pods="110"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850142    2552 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850147    2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850152    2552 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850158    2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850164    2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850169    2552 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850182    2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 18 01:48:14.857787 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850195    2552 flags.go:64] FLAG: --node-status-max-images="50"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850200    2552 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850206    2552 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850337    2552 flags.go:64] FLAG: --pod-cidr=""
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850363    2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850376    2552 flags.go:64] FLAG: --pod-manifest-path=""
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850382    2552 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850389    2552 flags.go:64] FLAG: --pods-per-core="0"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850395    2552 flags.go:64] FLAG: --port="10250"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850401    2552 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850407    2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e221d3474479ce0d"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850414    2552 flags.go:64] FLAG: --qos-reserved=""
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850444    2552 flags.go:64] FLAG: --read-only-port="10255"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850450    2552 flags.go:64] FLAG: --register-node="true"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850454    2552 flags.go:64] FLAG: --register-with-taints=""
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850462    2552 flags.go:64] FLAG: --registry-burst="10"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850466    2552 flags.go:64] FLAG: --registry-qps="5"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850471    2552 flags.go:64] FLAG: --reserved-cpus=""
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850476    2552 flags.go:64] FLAG: --reserved-memory=""
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850482    2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850492    2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850498    2552 flags.go:64] FLAG: --rotate-certificates="false"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850502    2552 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850507    2552 flags.go:64] FLAG: --runonce="false"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850512    2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 18 01:48:14.858681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850517    2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850531    2552 flags.go:64] FLAG: --seccomp-default="false"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850536    2552 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850546    2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850552    2552 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850557    2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850562    2552 flags.go:64] FLAG: --storage-driver-password="root"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850568    2552 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850573    2552 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850577    2552 flags.go:64] FLAG: --storage-driver-user="root"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850582    2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850589    2552 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850600    2552 flags.go:64] FLAG: --system-cgroups=""
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850605    2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850621    2552 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850628    2552 flags.go:64] FLAG: --tls-cert-file=""
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850633    2552 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850641    2552 flags.go:64] FLAG: --tls-min-version=""
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850646    2552 flags.go:64] FLAG: --tls-private-key-file=""
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850657    2552 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850662    2552 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850667    2552 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850672    2552 flags.go:64] FLAG: --v="2"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850683    2552 flags.go:64] FLAG: --version="false"
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850690    2552 flags.go:64] FLAG: --vmodule=""
Jul 18 01:48:14.859796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850695    2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.850701    2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851051    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851059    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851069    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851074    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851078    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851081    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851119    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851143    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851151    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851156    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851165    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851738    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851749    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851752    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851757    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851760    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851763    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:14.860866 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851767    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851770    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851773    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851776    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851778    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851781    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851784    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851786    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851788    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851791    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851794    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851796    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851799    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851801    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851804    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851806    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851809    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851811    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851814    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851817    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:14.861787 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851819    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851822    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851824    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851827    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851830    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851832    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851835    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851838    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851841    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851843    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851848    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851852    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851855    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851858    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851861    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851864    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851867    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851870    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851873    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:14.862408 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851875    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851878    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851880    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851883    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851885    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851888    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851892    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851895    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851897    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851900    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851903    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851905    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851908    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851910    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851912    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851915    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851917    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851920    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851923    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851926    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:14.863084 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851929    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851932    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851935    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851939    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851942    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851945    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851948    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851951    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851954    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851956    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851959    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851961    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851964    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851966    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851968    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851971    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851974    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851976    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851979    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851982    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:14.863924 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851985    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851987    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.851990    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.852537    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.860401    2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.860446    2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.860476    2552 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.860485    2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860586    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860596    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860601    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860606    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860610    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860614    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:14.864566 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860618    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860623    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860627    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860632    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860636    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860641    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860645    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860649    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860653    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860657    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860663    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860667    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860671    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860675    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860679    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860684    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860688    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860692    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860696    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860700    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:14.865047 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860704    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860708    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860713    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860719    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860723    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860727    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860731    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860735    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860740    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860744    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860749    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860754    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860758    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860762    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860766    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860770    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860775    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860779    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860783    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860787    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:14.865575 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860791    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860796    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860800    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860811    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860816    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860820    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860826    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860833    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860837    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860842    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860847    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860851    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860858    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860863    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860868    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860872    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860877    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860881    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860884    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:14.866055 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860888    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860892    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860897    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860901    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860905    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860909    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860913    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860918    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860922    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860926    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860930    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860936    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860942    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860946    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860951    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860955    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860960    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860965    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860970    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:14.866589 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860974    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860978    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860982    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860986    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860991    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860995    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.860999    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861003    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861007    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861012    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861016    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861021    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861025    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861030    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861034    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:14.867079 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.861042    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861268    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861277    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861282    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861287    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861292    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861296    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861300    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861304    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861310    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861316    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861321    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861326    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861330    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861334    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861338    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861343    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861347    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861352    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:48:14.867558 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861356    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861360    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861364    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861368    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861372    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861376    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861395    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861400    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861405    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861411    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861433    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861438    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861444    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861449    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861453    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861457    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861461    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861466    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861469    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:48:14.868034 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861473    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861477    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861480    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861484    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861488    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861491    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861495    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861499    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861502    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861505    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861510    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861514    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861519    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861523    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861528    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861532    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861536    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861540    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861544    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861549    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:48:14.868722 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861554    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861558    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861562    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861566    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861570    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861574    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861578    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861582    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861586    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861590    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861595    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861599    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861603    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861607    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861611    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861615    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861619    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861623    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861628    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861631    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:48:14.869582 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861636    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861639    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861644    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861648    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861653    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861660    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861664    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861668    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861672    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861676    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861681    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861685    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861690    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861694    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861698    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861702    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861706    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861710    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861715    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861719    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:48:14.870161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861723    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:48:14.870676 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:14.861728    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:48:14.870676 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.861736    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:48:14.870676 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.861934    2552 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 18 01:48:14.870676 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.864794    2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 18 01:48:14.870676 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.866125    2552 server.go:1019] "Starting client certificate rotation"
Jul 18 01:48:14.870676 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.866237    2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 01:48:14.870676 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.866276    2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 01:48:14.870676 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.868985    2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 01:48:14.872410 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.872389    2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 01:48:14.882967 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.882951    2552 log.go:25] "Validated CRI v1 runtime API"
Jul 18 01:48:14.884960 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.884945    2552 log.go:25] "Validated CRI v1 image API"
Jul 18 01:48:14.885765 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.885748    2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 18 01:48:14.888432 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.888390    2552 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ac7677de-d55c-4db6-a1c1-140a34e13cde:/dev/nvme0n1p4 fdf72111-5a57-498b-bf12-e51ae7296ede:/dev/nvme0n1p3]
Jul 18 01:48:14.888509 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.888417    2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 18 01:48:14.893973 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.893817    2552 manager.go:216] Machine: {Timestamp:2026-07-18 01:48:14.892716979 +0000 UTC m=+0.332213918 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098546 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b0fb410f446ce4be95143e3b4d4c1 SystemUUID:ec2b0fb4-10f4-46ce-4be9-5143e3b4d4c1 BootID:1c003390-8171-45ea-b2e8-7207695d8318 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6d:b1:f3:ee:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6d:b1:f3:ee:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:09:6a:75:3e:2d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 18 01:48:14.893973 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.893963    2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 18 01:48:14.894105 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.894049    2552 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 18 01:48:14.896169 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.896142    2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 18 01:48:14.896312 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.896174    2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-48.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 18 01:48:14.897037 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.897026    2552 topology_manager.go:138] "Creating topology manager with none policy"
Jul 18 01:48:14.897075 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.897040    2552 container_manager_linux.go:309] "Creating device plugin manager"
Jul 18 01:48:14.897075 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.897048    2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 01:48:14.897331 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.897317    2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 01:48:14.897801 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.897787    2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 01:48:14.899028 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.899017    2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 18 01:48:14.902547 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.902533    2552 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 01:48:14.902724 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.902714    2552 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 18 01:48:14.902766 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.902747    2552 kubelet.go:486] "Attempting to sync node with API server"
Jul 18 01:48:14.902766 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.902757    2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 18 01:48:14.902844 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.902775    2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 18 01:48:14.902844 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.902791    2552 kubelet.go:398] "Adding apiserver pod source"
Jul 18 01:48:14.902844 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.902799    2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 18 01:48:14.903208 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.903195    2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 01:48:14.903277 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.903214    2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 01:48:14.905511 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.905491    2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 18 01:48:14.907189 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.907176    2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 18 01:48:14.907228 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.907207    2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 18 01:48:14.908770 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908758    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 18 01:48:14.908809 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908782    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 18 01:48:14.908809 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908795    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 18 01:48:14.908809 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908802    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 18 01:48:14.908897 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908811    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 18 01:48:14.908897 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908816    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 18 01:48:14.908897 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908824    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 18 01:48:14.908897 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908840    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 18 01:48:14.908897 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908847    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 18 01:48:14.908897 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908853    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 18 01:48:14.908897 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908868    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 18 01:48:14.908897 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.908874    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 18 01:48:14.910313 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.910300    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 18 01:48:14.910313 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.910311    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 18 01:48:14.913515 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.913498    2552 server.go:1268] "Started kubelet"
Jul 18 01:48:14.913678 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.913612    2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 18 01:48:14.913830 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.913615    2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 18 01:48:14.913830 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.913741    2552 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 18 01:48:14.914873 ip-10-0-128-48 systemd[1]: Started Kubernetes Kubelet.
Jul 18 01:48:14.915563 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.915521    2552 server.go:310] "Adding debug handlers to kubelet server"
Jul 18 01:48:14.915671 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.915651    2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-48.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 18 01:48:14.915824 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:14.915666    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 18 01:48:14.915824 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:14.915787    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-48.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 18 01:48:14.918165 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.918144    2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 18 01:48:14.920143 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:14.920125    2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 18 01:48:14.920676 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:14.919505    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-48.ec2.internal.18c33f2288169040  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-48.ec2.internal,UID:ip-10-0-128-48.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-48.ec2.internal,},FirstTimestamp:2026-07-18 01:48:14.913466432 +0000 UTC m=+0.352963355,LastTimestamp:2026-07-18 01:48:14.913466432 +0000 UTC m=+0.352963355,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-48.ec2.internal,}"
Jul 18 01:48:14.923275 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.923257    2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 18 01:48:14.923347 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.923297    2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 18 01:48:14.923669 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:14.923565    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:14.923991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.923977    2552 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 18 01:48:14.923991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.923992    2552 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 18 01:48:14.924105 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.923992    2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 18 01:48:14.924105 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.924065    2552 reconstruct.go:97] "Volume reconstruction finished"
Jul 18 01:48:14.924105 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.924071    2552 reconciler.go:29] "Reconciler: start to sync state"
Jul 18 01:48:14.928056 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.928033    2552 factory.go:55] Registering systemd factory
Jul 18 01:48:14.928135 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.928074    2552 factory.go:223] Registration of the systemd container factory successfully
Jul 18 01:48:14.928442 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.928337    2552 factory.go:153] Registering CRI-O factory
Jul 18 01:48:14.928442 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.928352    2552 factory.go:223] Registration of the crio container factory successfully
Jul 18 01:48:14.928442 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.928406    2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 18 01:48:14.928750 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.928457    2552 factory.go:103] Registering Raw factory
Jul 18 01:48:14.928750 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.928472    2552 manager.go:1196] Started watching for new ooms in manager
Jul 18 01:48:14.929527 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.929501    2552 manager.go:349] Starting recovery of all containers
Jul 18 01:48:14.938592 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:14.938562    2552 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-48.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 18 01:48:14.938711 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:14.938676    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 18 01:48:14.940752 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.940737    2552 manager.go:354] Recovery completed
Jul 18 01:48:14.945211 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.945192    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:14.947067 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.947041    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:14.947130 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.947081    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:14.947130 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.947095    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:14.947637 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.947621    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:14.947698 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.947622    2552 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 18 01:48:14.947739 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.947695    2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 18 01:48:14.947739 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.947715    2552 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 01:48:14.948317 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.948299    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:14.948381 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.948334    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:14.948381 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.948347    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:14.949825 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.949777    2552 policy_none.go:49] "None policy: Start"
Jul 18 01:48:14.949825 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.949792    2552 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 18 01:48:14.949825 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.949801    2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 18 01:48:14.951622 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.951610    2552 policy_none.go:47] "Start"
Jul 18 01:48:14.955295 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.955172    2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-69bsj"
Jul 18 01:48:14.955681 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:14.955600    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-48.ec2.internal.18c33f228a174479  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-48.ec2.internal,UID:ip-10-0-128-48.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-48.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-48.ec2.internal,},FirstTimestamp:2026-07-18 01:48:14.947067001 +0000 UTC m=+0.386563924,LastTimestamp:2026-07-18 01:48:14.947067001 +0000 UTC m=+0.386563924,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-48.ec2.internal,}"
Jul 18 01:48:14.958631 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:14.958568    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-48.ec2.internal.18c33f228a179a94  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-48.ec2.internal,UID:ip-10-0-128-48.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-48.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-48.ec2.internal,},FirstTimestamp:2026-07-18 01:48:14.947089044 +0000 UTC m=+0.386585967,LastTimestamp:2026-07-18 01:48:14.947089044 +0000 UTC m=+0.386585967,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-48.ec2.internal,}"
Jul 18 01:48:14.964478 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:14.964459    2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-69bsj"
Jul 18 01:48:14.966230 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:14.966167    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-48.ec2.internal.18c33f228a17c206  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-48.ec2.internal,UID:ip-10-0-128-48.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-48.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-48.ec2.internal,},FirstTimestamp:2026-07-18 01:48:14.947099142 +0000 UTC m=+0.386596065,LastTimestamp:2026-07-18 01:48:14.947099142 +0000 UTC m=+0.386596065,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-48.ec2.internal,}"
Jul 18 01:48:15.002080 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.002056    2552 manager.go:337] "Starting Device Plugin manager"
Jul 18 01:48:15.002213 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.002093    2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 18 01:48:15.002213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.002104    2552 server.go:87] "Starting device plugin registration server"
Jul 18 01:48:15.002460 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.002447    2552 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 18 01:48:15.002506 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.002464    2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 18 01:48:15.002589 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.002559    2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 18 01:48:15.002645 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.002631    2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 18 01:48:15.002645 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.002637    2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 18 01:48:15.003193 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.003174    2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 18 01:48:15.003305 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.003227    2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:15.067834 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.067799    2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 18 01:48:15.069716 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.069075    2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 18 01:48:15.069716 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.069109    2552 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 18 01:48:15.069716 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.069134    2552 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 18 01:48:15.069716 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.069165    2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 18 01:48:15.071952 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.071927    2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:15.102684 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.102604    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:15.103547 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.103521    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:15.103655 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.103563    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:15.103655 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.103579    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:15.103655 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.103603    2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.114230 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.114207    2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.114304 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.114234    2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-48.ec2.internal\": node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:15.170028 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.169990    2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-48.ec2.internal"]
Jul 18 01:48:15.170274 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.170074    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:15.171814 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.171795    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:15.171875 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.171829    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:15.171875 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.171839    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:15.173931 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.173917    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:15.174448 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.174414    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.174527 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.174470    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:15.174649 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.174635    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:15.174707 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.174660    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:15.174707 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.174674    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:15.175127 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.175113    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:15.175183 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.175140    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:15.175183 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.175160    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:15.176718 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.176702    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.176795 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.176733    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:15.177388 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.177371    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:15.177477 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.177406    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:15.177477 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.177435    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:15.197548 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.197515    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:15.199150 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.199131    2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-48.ec2.internal\" not found" node="ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.205003 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.204983    2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-48.ec2.internal\" not found" node="ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.297824 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.297788    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:15.325747 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.325715    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00fb35e30223243945f067fa48061f7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal\" (UID: \"00fb35e30223243945f067fa48061f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.325820 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.325750    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/099b81222a3a57c11d9d757e4965a26f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-48.ec2.internal\" (UID: \"099b81222a3a57c11d9d757e4965a26f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.325820 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.325769    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00fb35e30223243945f067fa48061f7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal\" (UID: \"00fb35e30223243945f067fa48061f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.398090 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.398066    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:15.426684 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.426655    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00fb35e30223243945f067fa48061f7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal\" (UID: \"00fb35e30223243945f067fa48061f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.426745 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.426606    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00fb35e30223243945f067fa48061f7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal\" (UID: \"00fb35e30223243945f067fa48061f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.426745 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.426710    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00fb35e30223243945f067fa48061f7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal\" (UID: \"00fb35e30223243945f067fa48061f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.426806 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.426745    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/099b81222a3a57c11d9d757e4965a26f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-48.ec2.internal\" (UID: \"099b81222a3a57c11d9d757e4965a26f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.426806 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.426774    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/099b81222a3a57c11d9d757e4965a26f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-48.ec2.internal\" (UID: \"099b81222a3a57c11d9d757e4965a26f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.426868 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.426800    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00fb35e30223243945f067fa48061f7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal\" (UID: \"00fb35e30223243945f067fa48061f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.498767 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.498726    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:15.501901 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.501884    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.507540 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.507515    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:15.599436 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.599384    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:15.699969 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.699885    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:15.723370 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.723341    2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:15.800893 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.800858    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:15.865375 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.865339    2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 18 01:48:15.865841 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.865505    2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 18 01:48:15.865841 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.865520    2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 18 01:48:15.901669 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:15.901633    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:15.968730 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.968642    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 01:43:15 +0000 UTC" deadline="2027-12-24 14:50:18.473490613 +0000 UTC"
Jul 18 01:48:15.968730 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:15.968680    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12589h2m2.504814131s"
Jul 18 01:48:16.001760 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:16.001725    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:16.038469 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.038439    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:48:16.039758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.039733    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:48:16.039854 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.039777    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:48:16.039854 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.039797    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:48:16.075997 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:16.075965    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00fb35e30223243945f067fa48061f7a.slice/crio-d62aff68c2cd4c7b7c08d3df86e0baf6d454a7f0ebce83492f45a5b1d93658ab WatchSource:0}: Error finding container d62aff68c2cd4c7b7c08d3df86e0baf6d454a7f0ebce83492f45a5b1d93658ab: Status 404 returned error can't find the container with id d62aff68c2cd4c7b7c08d3df86e0baf6d454a7f0ebce83492f45a5b1d93658ab
Jul 18 01:48:16.076208 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:16.076188    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099b81222a3a57c11d9d757e4965a26f.slice/crio-252e3669d5bf3222f9ee75d0e64cbdaf9bbbb21fb2092434cc5b778a589a4c2a WatchSource:0}: Error finding container 252e3669d5bf3222f9ee75d0e64cbdaf9bbbb21fb2092434cc5b778a589a4c2a: Status 404 returned error can't find the container with id 252e3669d5bf3222f9ee75d0e64cbdaf9bbbb21fb2092434cc5b778a589a4c2a
Jul 18 01:48:16.080695 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.080679    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:48:16.101904 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:16.101874    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:16.202793 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:16.202761    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-48.ec2.internal\" not found"
Jul 18 01:48:16.233877 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.233799    2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:16.324985 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.324954    2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:16.345047 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.345023    2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 01:48:16.346232 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.346217    2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-48.ec2.internal"
Jul 18 01:48:16.357007 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.356987    2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 01:48:16.490736 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.490648    2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:16.904446 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.904403    2552 apiserver.go:52] "Watching apiserver"
Jul 18 01:48:16.911761 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.911729    2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 18 01:48:16.912245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.912223    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zr5mn","kube-system/konnectivity-agent-dmwh7","kube-system/kube-apiserver-proxy-ip-10-0-128-48.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz","openshift-image-registry/node-ca-nqcrm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal","openshift-multus/multus-additional-cni-plugins-7p4kc","openshift-network-diagnostics/network-check-target-42wx4","openshift-cluster-node-tuning-operator/tuned-67xc7","openshift-dns/node-resolver-strgq","openshift-multus/multus-kwdzd","openshift-multus/network-metrics-daemon-fgqqf","openshift-network-operator/iptables-alerter-8wrwt","openshift-ovn-kubernetes/ovnkube-node-nt7rh"]
Jul 18 01:48:16.915085 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.915056    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:16.917349 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.917318    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 18 01:48:16.917534 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.917513    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 18 01:48:16.917611 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.917534    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k64gx\""
Jul 18 01:48:16.917611 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.917539    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 18 01:48:16.917728 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.917625    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 18 01:48:16.917728 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.917660    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 18 01:48:16.918055 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.918036    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:16.920094 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.919945    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 18 01:48:16.920518 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.920495    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5k774\""
Jul 18 01:48:16.920792 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.920760    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 18 01:48:16.923078 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.923059    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.925200 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.924004    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:16.925200 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:16.924082    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:16.925200 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.925021    2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 18 01:48:16.925619 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.925439    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 18 01:48:16.925619 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.925481    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 18 01:48:16.926395 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.926374    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-22s56\""
Jul 18 01:48:16.926492 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.926422    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:16.926639 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.926525    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 18 01:48:16.926639 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.926616    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 18 01:48:16.926745 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:16.926694    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:16.926967 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.926950    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 18 01:48:16.927038 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.927014    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 18 01:48:16.929443 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.929410    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:16.931551 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.931532    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rx8t9\""
Jul 18 01:48:16.931893 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.931874    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nqcrm"
Jul 18 01:48:16.932528 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.932341    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 18 01:48:16.932528 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.932343    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 18 01:48:16.932691 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.932671    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 18 01:48:16.934071 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934044    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/422f305c-59b7-41b0-91bd-1fe60a29acda-cni-binary-copy\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:16.934155 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934084    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97cd8d7f-4245-4fad-984a-e68481681a9e-agent-certs\") pod \"konnectivity-agent-dmwh7\" (UID: \"97cd8d7f-4245-4fad-984a-e68481681a9e\") " pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:16.934155 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934110    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97cd8d7f-4245-4fad-984a-e68481681a9e-konnectivity-ca\") pod \"konnectivity-agent-dmwh7\" (UID: \"97cd8d7f-4245-4fad-984a-e68481681a9e\") " pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:16.934155 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934136    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00845976-489b-41e9-b40b-b4ee2d16a0b3-ovn-node-metrics-cert\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934280 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934163    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6l9f\" (UniqueName: \"kubernetes.io/projected/4de548f6-694f-425d-9ee4-5ac3df26aed4-kube-api-access-r6l9f\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:16.934280 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934189    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-system-cni-dir\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:16.934280 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934214    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:16.934437 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934240    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-systemd-units\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934437 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934313    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-node-log\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934437 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934336    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-cnibin\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:16.934437 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934362    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/422f305c-59b7-41b0-91bd-1fe60a29acda-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:16.934437 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934392    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cf4v\" (UniqueName: \"kubernetes.io/projected/422f305c-59b7-41b0-91bd-1fe60a29acda-kube-api-access-9cf4v\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:16.934437 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934419    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-run-systemd\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934442    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00845976-489b-41e9-b40b-b4ee2d16a0b3-env-overrides\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934455    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934466    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00845976-489b-41e9-b40b-b4ee2d16a0b3-ovnkube-script-lib\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934524    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-run-netns\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934551    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-etc-openvswitch\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934576    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-cni-netd\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934601    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-os-release\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934628    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-var-lib-openvswitch\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934653    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-run-ovn\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934677    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-log-socket\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934709    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934719    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934721    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfjwn\" (UniqueName: \"kubernetes.io/projected/00845976-489b-41e9-b40b-b4ee2d16a0b3-kube-api-access-mfjwn\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.934758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934759    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxq6c\" (UniqueName: \"kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c\") pod \"network-check-target-42wx4\" (UID: \"bb0b5223-b563-4075-8406-c096402a16c5\") " pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:16.935438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934789    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-slash\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.935438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934815    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.935438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934880    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:16.935438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934907    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-426dm\""
Jul 18 01:48:16.935438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934921    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/422f305c-59b7-41b0-91bd-1fe60a29acda-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:16.935438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934954    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-kubelet\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.935438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934956    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:16.935438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.934996    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-run-openvswitch\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.935438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.935027    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.935438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.935057    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-cni-bin\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.935438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.935081    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00845976-489b-41e9-b40b-b4ee2d16a0b3-ovnkube-config\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:16.937127 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.937042    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-strgq"
Jul 18 01:48:16.937912 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.937889    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:48:16.938086 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.938050    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qjmhb\""
Jul 18 01:48:16.938168 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.938104    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 18 01:48:16.939170 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.939150    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 18 01:48:16.939270 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.939226    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 18 01:48:16.939400 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.939384    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l7p4d\""
Jul 18 01:48:16.939481 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.939423    2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 01:48:16.939758 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.939730    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:16.941805 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.941782    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 18 01:48:16.942036 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.942019    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6rmd5\""
Jul 18 01:48:16.942808 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.942788    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:16.942894 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:16.942874    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:16.945099 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.945078    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8wrwt"
Jul 18 01:48:16.947218 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.947197    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8hnd4\""
Jul 18 01:48:16.947370 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.947351    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 18 01:48:16.947467 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.947434    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 18 01:48:16.947637 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.947616    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:48:16.962216 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.962187    2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4jsbh"
Jul 18 01:48:16.968770 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.968748    2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4jsbh"
Jul 18 01:48:16.991120 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:16.991093    2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:17.025031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.024990    2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 18 01:48:17.035615 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035580    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:17.035786 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035628    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-var-lib-cni-multus\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.035786 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035654    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-var-lib-kubelet\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.035786 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035676    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99vp9\" (UniqueName: \"kubernetes.io/projected/7518d4ee-c387-40a6-afce-250816fdd272-kube-api-access-99vp9\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.035786 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035701    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcpv5\" (UniqueName: \"kubernetes.io/projected/ab7b9b44-0deb-42a5-9bc6-26778ed68ee5-kube-api-access-vcpv5\") pod \"node-resolver-strgq\" (UID: \"ab7b9b44-0deb-42a5-9bc6-26778ed68ee5\") " pod="openshift-dns/node-resolver-strgq"
Jul 18 01:48:17.035786 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035738    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/422f305c-59b7-41b0-91bd-1fe60a29acda-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.035786 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035768    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-kubelet\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.036011 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035793    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-run-openvswitch\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.036011 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035819    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-cni-bin\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.036011 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035845    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/04de099d-ff55-45ed-bbc1-6ded34e65f0f-iptables-alerter-script\") pod \"iptables-alerter-8wrwt\" (UID: \"04de099d-ff55-45ed-bbc1-6ded34e65f0f\") " pod="openshift-network-operator/iptables-alerter-8wrwt"
Jul 18 01:48:17.036011 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035866    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-socket-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.036011 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035892    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-kubernetes\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.036011 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035912    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00845976-489b-41e9-b40b-b4ee2d16a0b3-ovnkube-config\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.036011 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035928    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg7jk\" (UniqueName: \"kubernetes.io/projected/04de099d-ff55-45ed-bbc1-6ded34e65f0f-kube-api-access-rg7jk\") pod \"iptables-alerter-8wrwt\" (UID: \"04de099d-ff55-45ed-bbc1-6ded34e65f0f\") " pod="openshift-network-operator/iptables-alerter-8wrwt"
Jul 18 01:48:17.036011 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035947    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-tuned\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.036011 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.035984    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97cd8d7f-4245-4fad-984a-e68481681a9e-agent-certs\") pod \"konnectivity-agent-dmwh7\" (UID: \"97cd8d7f-4245-4fad-984a-e68481681a9e\") " pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:17.036011 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036009    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04de099d-ff55-45ed-bbc1-6ded34e65f0f-host-slash\") pod \"iptables-alerter-8wrwt\" (UID: \"04de099d-ff55-45ed-bbc1-6ded34e65f0f\") " pod="openshift-network-operator/iptables-alerter-8wrwt"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036030    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-sys-fs\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036046    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5ac9a0d-d093-4e02-a9b2-402bddadb797-host\") pod \"node-ca-nqcrm\" (UID: \"c5ac9a0d-d093-4e02-a9b2-402bddadb797\") " pod="openshift-image-registry/node-ca-nqcrm"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036061    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-systemd\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036079    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-system-cni-dir\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036094    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-systemd-units\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036109    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-run-netns\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036125    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-cnibin\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036143    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/422f305c-59b7-41b0-91bd-1fe60a29acda-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036160    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cf4v\" (UniqueName: \"kubernetes.io/projected/422f305c-59b7-41b0-91bd-1fe60a29acda-kube-api-access-9cf4v\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036209    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-run-systemd\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036229    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00845976-489b-41e9-b40b-b4ee2d16a0b3-env-overrides\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036279    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00845976-489b-41e9-b40b-b4ee2d16a0b3-ovnkube-script-lib\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036300    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-cnibin\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036318    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-run-k8s-cni-cncf-io\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036335    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-etc-openvswitch\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036356    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-cni-dir\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.036408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036371    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-os-release\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036390    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbsw7\" (UniqueName: \"kubernetes.io/projected/8e638ffe-a9f0-453f-b92c-d9d3ee930270-kube-api-access-sbsw7\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036405    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45860c98-3146-470d-8c2c-3feb65e32e10-dbus\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036420    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz8kg\" (UniqueName: \"kubernetes.io/projected/c5ac9a0d-d093-4e02-a9b2-402bddadb797-kube-api-access-rz8kg\") pod \"node-ca-nqcrm\" (UID: \"c5ac9a0d-d093-4e02-a9b2-402bddadb797\") " pod="openshift-image-registry/node-ca-nqcrm"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036442    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-sysconfig\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036462    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxq6c\" (UniqueName: \"kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c\") pod \"network-check-target-42wx4\" (UID: \"bb0b5223-b563-4075-8406-c096402a16c5\") " pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036492    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-daemon-config\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036531    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-modprobe-d\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036554    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-sysctl-d\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036589    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-tmp\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036690    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-system-cni-dir\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036706    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-slash\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036749    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036785    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-system-cni-dir\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036814    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e638ffe-a9f0-453f-b92c-d9d3ee930270-cni-binary-copy\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036843    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-device-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036868    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-var-lib-kubelet\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.037103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036885    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab7b9b44-0deb-42a5-9bc6-26778ed68ee5-tmp-dir\") pod \"node-resolver-strgq\" (UID: \"ab7b9b44-0deb-42a5-9bc6-26778ed68ee5\") " pod="openshift-dns/node-resolver-strgq"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036906    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036923    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-registration-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036925    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-run-systemd\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036941    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-run-multus-certs\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036971    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-systemd-units\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036974    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-etc-kubernetes\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.036991    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-etc-selinux\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037008    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-sysctl-conf\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037025    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-run\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037041    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-sys\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037057    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-lib-modules\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037072    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ab7b9b44-0deb-42a5-9bc6-26778ed68ee5-hosts-file\") pod \"node-resolver-strgq\" (UID: \"ab7b9b44-0deb-42a5-9bc6-26778ed68ee5\") " pod="openshift-dns/node-resolver-strgq"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037092    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/422f305c-59b7-41b0-91bd-1fe60a29acda-cni-binary-copy\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037110    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97cd8d7f-4245-4fad-984a-e68481681a9e-konnectivity-ca\") pod \"konnectivity-agent-dmwh7\" (UID: \"97cd8d7f-4245-4fad-984a-e68481681a9e\") " pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037128    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00845976-489b-41e9-b40b-b4ee2d16a0b3-ovn-node-metrics-cert\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037146    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6l9f\" (UniqueName: \"kubernetes.io/projected/4de548f6-694f-425d-9ee4-5ac3df26aed4-kube-api-access-r6l9f\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:17.037712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037164    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-socket-dir-parent\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037226    2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037267    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45860c98-3146-470d-8c2c-3feb65e32e10-kubelet-config\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037286    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037304    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037310    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00845976-489b-41e9-b40b-b4ee2d16a0b3-ovnkube-config\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037320    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-node-log\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037310    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/422f305c-59b7-41b0-91bd-1fe60a29acda-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037339    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-var-lib-cni-bin\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037399    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-slash\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037400    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037447    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037806    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/422f305c-59b7-41b0-91bd-1fe60a29acda-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037855    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-hostroot\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037887    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037917    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5ac9a0d-d093-4e02-a9b2-402bddadb797-serviceca\") pod \"node-ca-nqcrm\" (UID: \"c5ac9a0d-d093-4e02-a9b2-402bddadb797\") " pod="openshift-image-registry/node-ca-nqcrm"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037930    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/422f305c-59b7-41b0-91bd-1fe60a29acda-cni-binary-copy\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.038245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037946    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-host\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.037991    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mwdx\" (UniqueName: \"kubernetes.io/projected/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-kube-api-access-9mwdx\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038019    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-conf-dir\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038053    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-run-netns\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038067    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97cd8d7f-4245-4fad-984a-e68481681a9e-konnectivity-ca\") pod \"konnectivity-agent-dmwh7\" (UID: \"97cd8d7f-4245-4fad-984a-e68481681a9e\") " pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038083    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-cni-netd\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038090    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038304    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-node-log\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038338    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-os-release\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038365    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-kubelet\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038376    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-var-lib-openvswitch\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038408    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-run-ovn\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038436    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-log-socket\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038466    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfjwn\" (UniqueName: \"kubernetes.io/projected/00845976-489b-41e9-b40b-b4ee2d16a0b3-kube-api-access-mfjwn\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038479    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-cnibin\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038542    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-etc-openvswitch\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038592    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-cni-bin\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.039260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038639    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-run-openvswitch\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.040150 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038672    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-var-lib-openvswitch\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.040150 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038688    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-cni-netd\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.040150 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038761    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-run-ovn\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.040150 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038765    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/422f305c-59b7-41b0-91bd-1fe60a29acda-os-release\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.040150 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038809    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-log-socket\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.040150 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.038926    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00845976-489b-41e9-b40b-b4ee2d16a0b3-host-run-netns\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.040150 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.039070    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00845976-489b-41e9-b40b-b4ee2d16a0b3-env-overrides\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.040150 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.039279    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00845976-489b-41e9-b40b-b4ee2d16a0b3-ovnkube-script-lib\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.040150 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.039342    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:17.040150 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.039400    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs podName:4de548f6-694f-425d-9ee4-5ac3df26aed4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:17.53938306 +0000 UTC m=+2.978879973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs") pod "network-metrics-daemon-fgqqf" (UID: "4de548f6-694f-425d-9ee4-5ac3df26aed4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:17.041459 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.041414    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97cd8d7f-4245-4fad-984a-e68481681a9e-agent-certs\") pod \"konnectivity-agent-dmwh7\" (UID: \"97cd8d7f-4245-4fad-984a-e68481681a9e\") " pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:17.042168 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.041622    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00845976-489b-41e9-b40b-b4ee2d16a0b3-ovn-node-metrics-cert\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.048449 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.042615    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:17.048449 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.042644    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:17.048449 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.042658    2552 projected.go:196] Error preparing data for projected volume kube-api-access-cxq6c for pod openshift-network-diagnostics/network-check-target-42wx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:17.048449 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.042724    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c podName:bb0b5223-b563-4075-8406-c096402a16c5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:17.542705236 +0000 UTC m=+2.982202152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cxq6c" (UniqueName: "kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c") pod "network-check-target-42wx4" (UID: "bb0b5223-b563-4075-8406-c096402a16c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:17.048449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.046847    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6l9f\" (UniqueName: \"kubernetes.io/projected/4de548f6-694f-425d-9ee4-5ac3df26aed4-kube-api-access-r6l9f\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:17.059398 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.059357    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfjwn\" (UniqueName: \"kubernetes.io/projected/00845976-489b-41e9-b40b-b4ee2d16a0b3-kube-api-access-mfjwn\") pod \"ovnkube-node-nt7rh\" (UID: \"00845976-489b-41e9-b40b-b4ee2d16a0b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.059864 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.059837    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cf4v\" (UniqueName: \"kubernetes.io/projected/422f305c-59b7-41b0-91bd-1fe60a29acda-kube-api-access-9cf4v\") pod \"multus-additional-cni-plugins-7p4kc\" (UID: \"422f305c-59b7-41b0-91bd-1fe60a29acda\") " pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.075424 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.075368    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-48.ec2.internal" event={"ID":"099b81222a3a57c11d9d757e4965a26f","Type":"ContainerStarted","Data":"252e3669d5bf3222f9ee75d0e64cbdaf9bbbb21fb2092434cc5b778a589a4c2a"}
Jul 18 01:48:17.076681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.076652    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal" event={"ID":"00fb35e30223243945f067fa48061f7a","Type":"ContainerStarted","Data":"d62aff68c2cd4c7b7c08d3df86e0baf6d454a7f0ebce83492f45a5b1d93658ab"}
Jul 18 01:48:17.138967 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.138934    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-system-cni-dir\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.138967 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.138972    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e638ffe-a9f0-453f-b92c-d9d3ee930270-cni-binary-copy\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.139191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.138998    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-device-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.139191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139015    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-var-lib-kubelet\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.139191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139036    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab7b9b44-0deb-42a5-9bc6-26778ed68ee5-tmp-dir\") pod \"node-resolver-strgq\" (UID: \"ab7b9b44-0deb-42a5-9bc6-26778ed68ee5\") " pod="openshift-dns/node-resolver-strgq"
Jul 18 01:48:17.139191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139063    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-registration-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.139191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139065    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-system-cni-dir\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.139191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139099    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-device-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.139191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139108    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-var-lib-kubelet\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.139191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139107    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-run-multus-certs\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.139191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139163    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-registration-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139199    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-run-multus-certs\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139238    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-etc-kubernetes\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139269    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-etc-selinux\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139297    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-sysctl-conf\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139320    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-etc-kubernetes\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139323    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-run\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139369    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-run\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139372    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-sys\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139406    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-lib-modules\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139412    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-etc-selinux\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139428    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-sys\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139411    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab7b9b44-0deb-42a5-9bc6-26778ed68ee5-tmp-dir\") pod \"node-resolver-strgq\" (UID: \"ab7b9b44-0deb-42a5-9bc6-26778ed68ee5\") " pod="openshift-dns/node-resolver-strgq"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139465    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ab7b9b44-0deb-42a5-9bc6-26778ed68ee5-hosts-file\") pod \"node-resolver-strgq\" (UID: \"ab7b9b44-0deb-42a5-9bc6-26778ed68ee5\") " pod="openshift-dns/node-resolver-strgq"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139500    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-socket-dir-parent\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139519    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45860c98-3146-470d-8c2c-3feb65e32e10-kubelet-config\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139520    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-sysctl-conf\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139512    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ab7b9b44-0deb-42a5-9bc6-26778ed68ee5-hosts-file\") pod \"node-resolver-strgq\" (UID: \"ab7b9b44-0deb-42a5-9bc6-26778ed68ee5\") " pod="openshift-dns/node-resolver-strgq"
Jul 18 01:48:17.139722 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139541    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139551    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-lib-modules\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139586    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-var-lib-cni-bin\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139591    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-socket-dir-parent\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139593    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45860c98-3146-470d-8c2c-3feb65e32e10-kubelet-config\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139621    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-var-lib-cni-bin\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139617    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-hostroot\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139649    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-hostroot\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139652    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e638ffe-a9f0-453f-b92c-d9d3ee930270-cni-binary-copy\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139661    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139588    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139700    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5ac9a0d-d093-4e02-a9b2-402bddadb797-serviceca\") pod \"node-ca-nqcrm\" (UID: \"c5ac9a0d-d093-4e02-a9b2-402bddadb797\") " pod="openshift-image-registry/node-ca-nqcrm"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139726    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-host\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139752    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mwdx\" (UniqueName: \"kubernetes.io/projected/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-kube-api-access-9mwdx\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139775    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-conf-dir\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.139783    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139804    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-host\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139821    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-var-lib-cni-multus\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.140449 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.139845    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret podName:45860c98-3146-470d-8c2c-3feb65e32e10 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:17.639824003 +0000 UTC m=+3.079320934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret") pod "global-pull-secret-syncer-zr5mn" (UID: "45860c98-3146-470d-8c2c-3feb65e32e10") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139855    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-var-lib-cni-multus\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139882    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-conf-dir\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139904    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-var-lib-kubelet\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139929    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99vp9\" (UniqueName: \"kubernetes.io/projected/7518d4ee-c387-40a6-afce-250816fdd272-kube-api-access-99vp9\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139951    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-var-lib-kubelet\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139955    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcpv5\" (UniqueName: \"kubernetes.io/projected/ab7b9b44-0deb-42a5-9bc6-26778ed68ee5-kube-api-access-vcpv5\") pod \"node-resolver-strgq\" (UID: \"ab7b9b44-0deb-42a5-9bc6-26778ed68ee5\") " pod="openshift-dns/node-resolver-strgq"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.139996    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/04de099d-ff55-45ed-bbc1-6ded34e65f0f-iptables-alerter-script\") pod \"iptables-alerter-8wrwt\" (UID: \"04de099d-ff55-45ed-bbc1-6ded34e65f0f\") " pod="openshift-network-operator/iptables-alerter-8wrwt"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140042    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-socket-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140069    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-kubernetes\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140102    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg7jk\" (UniqueName: \"kubernetes.io/projected/04de099d-ff55-45ed-bbc1-6ded34e65f0f-kube-api-access-rg7jk\") pod \"iptables-alerter-8wrwt\" (UID: \"04de099d-ff55-45ed-bbc1-6ded34e65f0f\") " pod="openshift-network-operator/iptables-alerter-8wrwt"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140150    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5ac9a0d-d093-4e02-a9b2-402bddadb797-serviceca\") pod \"node-ca-nqcrm\" (UID: \"c5ac9a0d-d093-4e02-a9b2-402bddadb797\") " pod="openshift-image-registry/node-ca-nqcrm"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140182    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-socket-dir\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140152    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-tuned\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140231    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-kubernetes\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140238    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04de099d-ff55-45ed-bbc1-6ded34e65f0f-host-slash\") pod \"iptables-alerter-8wrwt\" (UID: \"04de099d-ff55-45ed-bbc1-6ded34e65f0f\") " pod="openshift-network-operator/iptables-alerter-8wrwt"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140264    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-sys-fs\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.141524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140286    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5ac9a0d-d093-4e02-a9b2-402bddadb797-host\") pod \"node-ca-nqcrm\" (UID: \"c5ac9a0d-d093-4e02-a9b2-402bddadb797\") " pod="openshift-image-registry/node-ca-nqcrm"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140309    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-systemd\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140339    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-run-netns\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140347    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7518d4ee-c387-40a6-afce-250816fdd272-sys-fs\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140377    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04de099d-ff55-45ed-bbc1-6ded34e65f0f-host-slash\") pod \"iptables-alerter-8wrwt\" (UID: \"04de099d-ff55-45ed-bbc1-6ded34e65f0f\") " pod="openshift-network-operator/iptables-alerter-8wrwt"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140379    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-cnibin\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140403    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-run-netns\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140405    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-run-k8s-cni-cncf-io\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140421    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-systemd\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140441    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-cni-dir\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140465    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-os-release\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140469    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-cnibin\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140485    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-host-run-k8s-cni-cncf-io\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140469    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5ac9a0d-d093-4e02-a9b2-402bddadb797-host\") pod \"node-ca-nqcrm\" (UID: \"c5ac9a0d-d093-4e02-a9b2-402bddadb797\") " pod="openshift-image-registry/node-ca-nqcrm"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140509    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbsw7\" (UniqueName: \"kubernetes.io/projected/8e638ffe-a9f0-453f-b92c-d9d3ee930270-kube-api-access-sbsw7\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140535    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45860c98-3146-470d-8c2c-3feb65e32e10-dbus\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140552    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-os-release\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140559    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz8kg\" (UniqueName: \"kubernetes.io/projected/c5ac9a0d-d093-4e02-a9b2-402bddadb797-kube-api-access-rz8kg\") pod \"node-ca-nqcrm\" (UID: \"c5ac9a0d-d093-4e02-a9b2-402bddadb797\") " pod="openshift-image-registry/node-ca-nqcrm"
Jul 18 01:48:17.142367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140586    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-sysconfig\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140601    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/04de099d-ff55-45ed-bbc1-6ded34e65f0f-iptables-alerter-script\") pod \"iptables-alerter-8wrwt\" (UID: \"04de099d-ff55-45ed-bbc1-6ded34e65f0f\") " pod="openshift-network-operator/iptables-alerter-8wrwt"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140612    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-cni-dir\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140630    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45860c98-3146-470d-8c2c-3feb65e32e10-dbus\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140647    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-sysconfig\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140736    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-daemon-config\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140755    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-modprobe-d\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140771    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-sysctl-d\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140788    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-tmp\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140875    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-sysctl-d\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.140905    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-modprobe-d\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.141382    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8e638ffe-a9f0-453f-b92c-d9d3ee930270-multus-daemon-config\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.143109    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-etc-tuned\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.143244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.143163    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-tmp\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.151071 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.151013    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mwdx\" (UniqueName: \"kubernetes.io/projected/0b71a3cc-92ec-42ab-a331-aa067db2e9fb-kube-api-access-9mwdx\") pod \"tuned-67xc7\" (UID: \"0b71a3cc-92ec-42ab-a331-aa067db2e9fb\") " pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.151764 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.151736    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz8kg\" (UniqueName: \"kubernetes.io/projected/c5ac9a0d-d093-4e02-a9b2-402bddadb797-kube-api-access-rz8kg\") pod \"node-ca-nqcrm\" (UID: \"c5ac9a0d-d093-4e02-a9b2-402bddadb797\") " pod="openshift-image-registry/node-ca-nqcrm"
Jul 18 01:48:17.152087 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.152060    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcpv5\" (UniqueName: \"kubernetes.io/projected/ab7b9b44-0deb-42a5-9bc6-26778ed68ee5-kube-api-access-vcpv5\") pod \"node-resolver-strgq\" (UID: \"ab7b9b44-0deb-42a5-9bc6-26778ed68ee5\") " pod="openshift-dns/node-resolver-strgq"
Jul 18 01:48:17.152346 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.152327    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg7jk\" (UniqueName: \"kubernetes.io/projected/04de099d-ff55-45ed-bbc1-6ded34e65f0f-kube-api-access-rg7jk\") pod \"iptables-alerter-8wrwt\" (UID: \"04de099d-ff55-45ed-bbc1-6ded34e65f0f\") " pod="openshift-network-operator/iptables-alerter-8wrwt"
Jul 18 01:48:17.154188 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.154161    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbsw7\" (UniqueName: \"kubernetes.io/projected/8e638ffe-a9f0-453f-b92c-d9d3ee930270-kube-api-access-sbsw7\") pod \"multus-kwdzd\" (UID: \"8e638ffe-a9f0-453f-b92c-d9d3ee930270\") " pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.155646 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.155594    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99vp9\" (UniqueName: \"kubernetes.io/projected/7518d4ee-c387-40a6-afce-250816fdd272-kube-api-access-99vp9\") pod \"aws-ebs-csi-driver-node-xvrsz\" (UID: \"7518d4ee-c387-40a6-afce-250816fdd272\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.237697 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.237659    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7p4kc"
Jul 18 01:48:17.246646 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.246616    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:17.255764 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.255739    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:17.262861 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.262837    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz"
Jul 18 01:48:17.270765 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.270742    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nqcrm"
Jul 18 01:48:17.279962 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.279940    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-67xc7"
Jul 18 01:48:17.286824 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.286802    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-strgq"
Jul 18 01:48:17.294300 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.294273    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kwdzd"
Jul 18 01:48:17.302396 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.302375    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8wrwt"
Jul 18 01:48:17.545091 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.545054    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:17.545285 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.545123    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxq6c\" (UniqueName: \"kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c\") pod \"network-check-target-42wx4\" (UID: \"bb0b5223-b563-4075-8406-c096402a16c5\") " pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:17.545285 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.545255    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:17.545285 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.545253    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:17.545285 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.545274    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:17.545285 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.545288    2552 projected.go:196] Error preparing data for projected volume kube-api-access-cxq6c for pod openshift-network-diagnostics/network-check-target-42wx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:17.545610 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.545333    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs podName:4de548f6-694f-425d-9ee4-5ac3df26aed4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:18.54531452 +0000 UTC m=+3.984811458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs") pod "network-metrics-daemon-fgqqf" (UID: "4de548f6-694f-425d-9ee4-5ac3df26aed4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:17.545610 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.545352    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c podName:bb0b5223-b563-4075-8406-c096402a16c5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:18.545344138 +0000 UTC m=+3.984841049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxq6c" (UniqueName: "kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c") pod "network-check-target-42wx4" (UID: "bb0b5223-b563-4075-8406-c096402a16c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:17.645952 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.645913    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:17.646128 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.646055    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:17.646128 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:17.646119    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret podName:45860c98-3146-470d-8c2c-3feb65e32e10 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:18.646100962 +0000 UTC m=+4.085597875 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret") pod "global-pull-secret-syncer-zr5mn" (UID: "45860c98-3146-470d-8c2c-3feb65e32e10") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:17.971925 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.971861    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:43:17 +0000 UTC" deadline="2028-03-07 16:46:22.544970691 +0000 UTC"
Jul 18 01:48:17.971925 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:17.971918    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14366h58m4.57305829s"
Jul 18 01:48:18.116229 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:18.115875    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b71a3cc_92ec_42ab_a331_aa067db2e9fb.slice/crio-4a048bc0e656e1f154015ba5f322cf4a7c19cfe0d6c981cf93518c6b6c57b5c3 WatchSource:0}: Error finding container 4a048bc0e656e1f154015ba5f322cf4a7c19cfe0d6c981cf93518c6b6c57b5c3: Status 404 returned error can't find the container with id 4a048bc0e656e1f154015ba5f322cf4a7c19cfe0d6c981cf93518c6b6c57b5c3
Jul 18 01:48:18.121022 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:18.120963    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ac9a0d_d093_4e02_a9b2_402bddadb797.slice/crio-1cca76732063c7c07185ac6a40679d4643bf9d82040fbdfe7e0763962f62e2b6 WatchSource:0}: Error finding container 1cca76732063c7c07185ac6a40679d4643bf9d82040fbdfe7e0763962f62e2b6: Status 404 returned error can't find the container with id 1cca76732063c7c07185ac6a40679d4643bf9d82040fbdfe7e0763962f62e2b6
Jul 18 01:48:18.122828 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:18.122783    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e638ffe_a9f0_453f_b92c_d9d3ee930270.slice/crio-dbf392c8ed4adb01711682d94402dd0265f42877820fde27b37e9fef70ca97f3 WatchSource:0}: Error finding container dbf392c8ed4adb01711682d94402dd0265f42877820fde27b37e9fef70ca97f3: Status 404 returned error can't find the container with id dbf392c8ed4adb01711682d94402dd0265f42877820fde27b37e9fef70ca97f3
Jul 18 01:48:18.124054 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:18.124022    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00845976_489b_41e9_b40b_b4ee2d16a0b3.slice/crio-45cdbbc6554c9953da2b6960e902651e42bceff1a49738dca91fd2ee1a3a3e13 WatchSource:0}: Error finding container 45cdbbc6554c9953da2b6960e902651e42bceff1a49738dca91fd2ee1a3a3e13: Status 404 returned error can't find the container with id 45cdbbc6554c9953da2b6960e902651e42bceff1a49738dca91fd2ee1a3a3e13
Jul 18 01:48:18.124987 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:18.124923    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97cd8d7f_4245_4fad_984a_e68481681a9e.slice/crio-d15abe33d62165f8430066ca92fa8e7efaa96aa605590b7aeff906ce094ec2bf WatchSource:0}: Error finding container d15abe33d62165f8430066ca92fa8e7efaa96aa605590b7aeff906ce094ec2bf: Status 404 returned error can't find the container with id d15abe33d62165f8430066ca92fa8e7efaa96aa605590b7aeff906ce094ec2bf
Jul 18 01:48:18.553527 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:18.553120    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxq6c\" (UniqueName: \"kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c\") pod \"network-check-target-42wx4\" (UID: \"bb0b5223-b563-4075-8406-c096402a16c5\") " pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:18.553527 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:18.553451    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:18.553527 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:18.553480    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:18.553527 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:18.553527    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:18.553908 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:18.553544    2552 projected.go:196] Error preparing data for projected volume kube-api-access-cxq6c for pod openshift-network-diagnostics/network-check-target-42wx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:18.553908 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:18.553578    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:18.553908 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:18.553613    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c podName:bb0b5223-b563-4075-8406-c096402a16c5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:20.553589529 +0000 UTC m=+5.993086444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxq6c" (UniqueName: "kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c") pod "network-check-target-42wx4" (UID: "bb0b5223-b563-4075-8406-c096402a16c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:18.553908 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:18.553638    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs podName:4de548f6-694f-425d-9ee4-5ac3df26aed4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:20.553621004 +0000 UTC m=+5.993117938 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs") pod "network-metrics-daemon-fgqqf" (UID: "4de548f6-694f-425d-9ee4-5ac3df26aed4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:18.654796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:18.654747    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:18.654984 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:18.654933    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:18.655050 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:18.655005    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret podName:45860c98-3146-470d-8c2c-3feb65e32e10 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:20.654983837 +0000 UTC m=+6.094480969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret") pod "global-pull-secret-syncer-zr5mn" (UID: "45860c98-3146-470d-8c2c-3feb65e32e10") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:18.947173 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:18.946919    2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:48:18.972649 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:18.972592    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:43:17 +0000 UTC" deadline="2027-12-20 21:24:51.051048815 +0000 UTC"
Jul 18 01:48:18.972649 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:18.972625    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12499h36m32.078428477s"
Jul 18 01:48:19.072914 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.072876    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:19.073101 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:19.073020    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:19.073459 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.073437    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:19.073588 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:19.073563    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:19.073663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.073648    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:19.073764 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:19.073745    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:19.107815 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.107700    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nqcrm" event={"ID":"c5ac9a0d-d093-4e02-a9b2-402bddadb797","Type":"ContainerStarted","Data":"1cca76732063c7c07185ac6a40679d4643bf9d82040fbdfe7e0763962f62e2b6"}
Jul 18 01:48:19.125524 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.125469    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-48.ec2.internal" event={"ID":"099b81222a3a57c11d9d757e4965a26f","Type":"ContainerStarted","Data":"67091587a8e7a060338b077749ef9232bcf181587b510c3a39ba6a380d8eaac1"}
Jul 18 01:48:19.136535 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.136445    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-strgq" event={"ID":"ab7b9b44-0deb-42a5-9bc6-26778ed68ee5","Type":"ContainerStarted","Data":"a1b5cb80b03306b50ba89856493aa7a209cd26679eab25aba5fe73673adc7b5a"}
Jul 18 01:48:19.146393 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.143659    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-48.ec2.internal" podStartSLOduration=3.143609006 podStartE2EDuration="3.143609006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:19.143486143 +0000 UTC m=+4.582983070" watchObservedRunningTime="2026-07-18 01:48:19.143609006 +0000 UTC m=+4.583105939"
Jul 18 01:48:19.146641 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.146612    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz" event={"ID":"7518d4ee-c387-40a6-afce-250816fdd272","Type":"ContainerStarted","Data":"9534f03550dd1a7723e49732e57449e69563c25ccb4e90dcd6dbe774fe629de1"}
Jul 18 01:48:19.154598 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.154436    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8wrwt" event={"ID":"04de099d-ff55-45ed-bbc1-6ded34e65f0f","Type":"ContainerStarted","Data":"f4cea7b34b74f34bf44ebd60222a930f0dfeed4b73b314e436483f679eaed2d8"}
Jul 18 01:48:19.156322 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.156296    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh" event={"ID":"00845976-489b-41e9-b40b-b4ee2d16a0b3","Type":"ContainerStarted","Data":"45cdbbc6554c9953da2b6960e902651e42bceff1a49738dca91fd2ee1a3a3e13"}
Jul 18 01:48:19.168271 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.168203    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kwdzd" event={"ID":"8e638ffe-a9f0-453f-b92c-d9d3ee930270","Type":"ContainerStarted","Data":"dbf392c8ed4adb01711682d94402dd0265f42877820fde27b37e9fef70ca97f3"}
Jul 18 01:48:19.177485 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.177425    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-67xc7" event={"ID":"0b71a3cc-92ec-42ab-a331-aa067db2e9fb","Type":"ContainerStarted","Data":"4a048bc0e656e1f154015ba5f322cf4a7c19cfe0d6c981cf93518c6b6c57b5c3"}
Jul 18 01:48:19.190998 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.190954    2552 generic.go:356] "Generic (PLEG): container finished" podID="00fb35e30223243945f067fa48061f7a" containerID="8008591de004e1f56ac01e7443cb170f29ee27a7499a0f2c0f506dd34e4eb6a5" exitCode=0
Jul 18 01:48:19.191313 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.191280    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal" event={"ID":"00fb35e30223243945f067fa48061f7a","Type":"ContainerDied","Data":"8008591de004e1f56ac01e7443cb170f29ee27a7499a0f2c0f506dd34e4eb6a5"}
Jul 18 01:48:19.201601 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.201477    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p4kc" event={"ID":"422f305c-59b7-41b0-91bd-1fe60a29acda","Type":"ContainerStarted","Data":"4bc4e430eb71b1bcee6223fb4941de76cd4b47c4156fcdea90d1edd3b550c65e"}
Jul 18 01:48:19.211203 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:19.211118    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dmwh7" event={"ID":"97cd8d7f-4245-4fad-984a-e68481681a9e","Type":"ContainerStarted","Data":"d15abe33d62165f8430066ca92fa8e7efaa96aa605590b7aeff906ce094ec2bf"}
Jul 18 01:48:20.255611 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:20.255531    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal" event={"ID":"00fb35e30223243945f067fa48061f7a","Type":"ContainerStarted","Data":"4e5a8b6cae4d4bd199fea824f4536e3b43baf5d270207d39e1186f0a80a7ae8a"}
Jul 18 01:48:20.273189 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:20.272150    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-48.ec2.internal" podStartSLOduration=4.272127809 podStartE2EDuration="4.272127809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:20.271028616 +0000 UTC m=+5.710525552" watchObservedRunningTime="2026-07-18 01:48:20.272127809 +0000 UTC m=+5.711624747"
Jul 18 01:48:20.571926 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:20.571828    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxq6c\" (UniqueName: \"kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c\") pod \"network-check-target-42wx4\" (UID: \"bb0b5223-b563-4075-8406-c096402a16c5\") " pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:20.571926 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:20.571924    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:20.572169 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:20.572060    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:20.572169 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:20.572126    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs podName:4de548f6-694f-425d-9ee4-5ac3df26aed4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:24.572106755 +0000 UTC m=+10.011603682 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs") pod "network-metrics-daemon-fgqqf" (UID: "4de548f6-694f-425d-9ee4-5ac3df26aed4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:20.572597 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:20.572565    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:20.572597 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:20.572587    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:20.572739 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:20.572601    2552 projected.go:196] Error preparing data for projected volume kube-api-access-cxq6c for pod openshift-network-diagnostics/network-check-target-42wx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:20.572739 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:20.572648    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c podName:bb0b5223-b563-4075-8406-c096402a16c5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:24.572632302 +0000 UTC m=+10.012129230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxq6c" (UniqueName: "kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c") pod "network-check-target-42wx4" (UID: "bb0b5223-b563-4075-8406-c096402a16c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:20.672418 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:20.672328    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:20.672625 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:20.672496    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:20.672625 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:20.672570    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret podName:45860c98-3146-470d-8c2c-3feb65e32e10 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:24.672546452 +0000 UTC m=+10.112043376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret") pod "global-pull-secret-syncer-zr5mn" (UID: "45860c98-3146-470d-8c2c-3feb65e32e10") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:21.069820 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:21.069783    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:21.070038 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:21.069942    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:21.070389 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:21.070364    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:21.070545 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:21.070472    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:21.071001 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:21.070791    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:21.071001 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:21.070890    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:23.073155 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:23.072173    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:23.073155 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:23.072319    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:23.073155 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:23.072729    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:23.073155 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:23.072830    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:23.073155 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:23.072881    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:23.073155 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:23.072962    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:24.605136 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:24.605088    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:24.605631 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:24.605154    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxq6c\" (UniqueName: \"kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c\") pod \"network-check-target-42wx4\" (UID: \"bb0b5223-b563-4075-8406-c096402a16c5\") " pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:24.605631 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:24.605309    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:24.605631 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:24.605326    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:24.605631 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:24.605340    2552 projected.go:196] Error preparing data for projected volume kube-api-access-cxq6c for pod openshift-network-diagnostics/network-check-target-42wx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:24.605631 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:24.605397    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c podName:bb0b5223-b563-4075-8406-c096402a16c5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:32.605379359 +0000 UTC m=+18.044876290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxq6c" (UniqueName: "kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c") pod "network-check-target-42wx4" (UID: "bb0b5223-b563-4075-8406-c096402a16c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:24.605891 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:24.605800    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:24.605891 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:24.605849    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs podName:4de548f6-694f-425d-9ee4-5ac3df26aed4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:32.605835506 +0000 UTC m=+18.045332417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs") pod "network-metrics-daemon-fgqqf" (UID: "4de548f6-694f-425d-9ee4-5ac3df26aed4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:24.706152 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:24.706063    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:24.706315 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:24.706199    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:24.706315 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:24.706267    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret podName:45860c98-3146-470d-8c2c-3feb65e32e10 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:32.706249952 +0000 UTC m=+18.145746882 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret") pod "global-pull-secret-syncer-zr5mn" (UID: "45860c98-3146-470d-8c2c-3feb65e32e10") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:25.070586 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:25.070548    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:25.070778 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:25.070720    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:25.072327 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:25.072173    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:25.072327 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:25.072268    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:25.072532 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:25.072337    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:25.072532 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:25.072468    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:27.069720 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:27.069687    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:27.070186 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:27.069828    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:27.070255 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:27.070205    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:27.070306 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:27.070283    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:27.070383 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:27.070336    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:27.070463 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:27.070407    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:29.069697 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:29.069655    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:29.069697 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:29.069678    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:29.070239 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:29.069655    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:29.070239 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:29.069790    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:29.070239 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:29.069862    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:29.070239 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:29.069938    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:31.070212 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:31.070176    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:31.070713 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:31.070176    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:31.070713 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:31.070309    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:31.070713 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:31.070332    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:31.070713 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:31.070414    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:31.070713 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:31.070548    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:32.665516 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:32.665480    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxq6c\" (UniqueName: \"kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c\") pod \"network-check-target-42wx4\" (UID: \"bb0b5223-b563-4075-8406-c096402a16c5\") " pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:32.665982 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:32.665549    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:32.665982 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:32.665666    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:32.665982 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:32.665690    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:32.665982 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:32.665714    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:32.665982 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:32.665728    2552 projected.go:196] Error preparing data for projected volume kube-api-access-cxq6c for pod openshift-network-diagnostics/network-check-target-42wx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:32.665982 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:32.665741    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs podName:4de548f6-694f-425d-9ee4-5ac3df26aed4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:48.665719703 +0000 UTC m=+34.105216626 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs") pod "network-metrics-daemon-fgqqf" (UID: "4de548f6-694f-425d-9ee4-5ac3df26aed4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:32.665982 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:32.665777    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c podName:bb0b5223-b563-4075-8406-c096402a16c5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:48.665761553 +0000 UTC m=+34.105258465 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxq6c" (UniqueName: "kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c") pod "network-check-target-42wx4" (UID: "bb0b5223-b563-4075-8406-c096402a16c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:32.766823 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:32.766783    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:32.766991 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:32.766963    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:32.767056 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:32.767045    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret podName:45860c98-3146-470d-8c2c-3feb65e32e10 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:48.767023166 +0000 UTC m=+34.206520095 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret") pod "global-pull-secret-syncer-zr5mn" (UID: "45860c98-3146-470d-8c2c-3feb65e32e10") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:33.069613 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:33.069528    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:33.069810 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:33.069647    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:33.070047 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:33.070027    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:33.070137 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:33.070122    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:33.070208 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:33.070193    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:33.070297 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:33.070282    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:35.070305 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:35.070262    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:35.070822 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:35.070384    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:35.070822 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:35.070470    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:35.070822 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:35.070562    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:35.070822 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:35.070587    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:35.070822 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:35.070626    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:37.069861 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:37.069816    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:37.070485 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:37.069819    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:37.070485 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:37.069956    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:37.070485 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:37.069829    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:37.070485 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:37.070036    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:37.070485 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:37.070131    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:38.294918 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.294279    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-strgq" event={"ID":"ab7b9b44-0deb-42a5-9bc6-26778ed68ee5","Type":"ContainerStarted","Data":"c6a1f601796cba233117c50cd06725b9923abaa2cb988089532beffaf7ebdbd3"}
Jul 18 01:48:38.296493 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.296243    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz" event={"ID":"7518d4ee-c387-40a6-afce-250816fdd272","Type":"ContainerStarted","Data":"134e6849198270001ebfc67602c3e504eb020c6fdf98ee6a3ed5ce4269f9eab3"}
Jul 18 01:48:38.299624 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.299597    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh" event={"ID":"00845976-489b-41e9-b40b-b4ee2d16a0b3","Type":"ContainerStarted","Data":"cc538191cc594380ab84f1a1ece23c9b88b4fcda88ac62bdccf9e23d04e7ad4e"}
Jul 18 01:48:38.299735 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.299677    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh" event={"ID":"00845976-489b-41e9-b40b-b4ee2d16a0b3","Type":"ContainerStarted","Data":"432d2da7198a25ede9ac503f88bd82929426631d9aa35aec77acf287944d5bf6"}
Jul 18 01:48:38.299796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.299742    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh" event={"ID":"00845976-489b-41e9-b40b-b4ee2d16a0b3","Type":"ContainerStarted","Data":"3a9a467804d61d4a9d814191307e688634267b075bb0e1156586ba6d1f6e4313"}
Jul 18 01:48:38.299796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.299791    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh" event={"ID":"00845976-489b-41e9-b40b-b4ee2d16a0b3","Type":"ContainerStarted","Data":"3ba4a935b46a6fbf4ae0ee6035fc33a9cae6b9e734193973b60b645e96d44441"}
Jul 18 01:48:38.299900 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.299838    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh" event={"ID":"00845976-489b-41e9-b40b-b4ee2d16a0b3","Type":"ContainerStarted","Data":"0a7cfc922e5e8615bda48e00cd75746ca07530ce604fabf1cdbb05cd05b87050"}
Jul 18 01:48:38.301483 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.301462    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kwdzd" event={"ID":"8e638ffe-a9f0-453f-b92c-d9d3ee930270","Type":"ContainerStarted","Data":"c8c2bacf52adce291a4ab2e5e3bbbfbdf0ef28bcffe01ed37a105ef9f64633a0"}
Jul 18 01:48:38.303132 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.303112    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-67xc7" event={"ID":"0b71a3cc-92ec-42ab-a331-aa067db2e9fb","Type":"ContainerStarted","Data":"29e26e9118094fab187b33a83f0035e45dc7fb871b05180d5c8cee387a849c1d"}
Jul 18 01:48:38.305066 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.305040    2552 generic.go:356] "Generic (PLEG): container finished" podID="422f305c-59b7-41b0-91bd-1fe60a29acda" containerID="0976baa596470555388bca578d18f0e197d0ba28464c2effd6896162dd9fabb1" exitCode=0
Jul 18 01:48:38.305327 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.305300    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p4kc" event={"ID":"422f305c-59b7-41b0-91bd-1fe60a29acda","Type":"ContainerDied","Data":"0976baa596470555388bca578d18f0e197d0ba28464c2effd6896162dd9fabb1"}
Jul 18 01:48:38.307741 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.307699    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-strgq" podStartSLOduration=3.016834689 podStartE2EDuration="22.307686194s" totalImagesPullingTime="19.290851505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:38.307009443 +0000 UTC m=+23.746506368" watchObservedRunningTime="2026-07-18 01:48:38.307686194 +0000 UTC m=+23.747183127"
Jul 18 01:48:38.311008 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.310615    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dmwh7" event={"ID":"97cd8d7f-4245-4fad-984a-e68481681a9e","Type":"ContainerStarted","Data":"9d5d1c190be30b497f63e5d9f1b8e8528622c35b441805c7fc4fb5bc44447643"}
Jul 18 01:48:38.314084 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.314051    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nqcrm" event={"ID":"c5ac9a0d-d093-4e02-a9b2-402bddadb797","Type":"ContainerStarted","Data":"664eda08d1b473eff0fba90e0edbc6b9735130c226567ea85bafa5d08f43005c"}
Jul 18 01:48:38.348674 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.348618    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-67xc7" podStartSLOduration=3.002327858 podStartE2EDuration="22.348599634s" totalImagesPullingTime="19.346271776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:38.348317703 +0000 UTC m=+23.787814636" watchObservedRunningTime="2026-07-18 01:48:38.348599634 +0000 UTC m=+23.788096565"
Jul 18 01:48:38.369007 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.368943    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kwdzd" podStartSLOduration=2.977831331 podStartE2EDuration="22.368921115s" totalImagesPullingTime="19.391089784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:38.368590057 +0000 UTC m=+23.808086990" watchObservedRunningTime="2026-07-18 01:48:38.368921115 +0000 UTC m=+23.808418050"
Jul 18 01:48:38.444652 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:38.444590    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dmwh7" podStartSLOduration=7.917185949 podStartE2EDuration="23.44457161s" totalImagesPullingTime="15.527385661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:38.410488447 +0000 UTC m=+23.849985399" watchObservedRunningTime="2026-07-18 01:48:38.44457161 +0000 UTC m=+23.884068542"
Jul 18 01:48:39.070941 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:39.070915    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:39.071126 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:39.071023    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:39.071358 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:39.071342    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:39.071450 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:39.071415    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:39.071511 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:39.071499    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:39.071597 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:39.071582    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:39.153391 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:39.153353    2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 18 01:48:39.318185 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:39.318097    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8wrwt" event={"ID":"04de099d-ff55-45ed-bbc1-6ded34e65f0f","Type":"ContainerStarted","Data":"12bff2eeaca7878f147e869fe416799c2083e099b37bb3fbdac2dc791b977a92"}
Jul 18 01:48:39.321572 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:39.321541    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh" event={"ID":"00845976-489b-41e9-b40b-b4ee2d16a0b3","Type":"ContainerStarted","Data":"4d1e6e5e21e12e1e1262c744c61057e8ae46c6d6120124035e096da31f0437c8"}
Jul 18 01:48:39.323642 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:39.323581    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz" event={"ID":"7518d4ee-c387-40a6-afce-250816fdd272","Type":"ContainerStarted","Data":"e8593d603ef68122a84221fce100e635c210c47e80decbd2072b1b33fc01651f"}
Jul 18 01:48:39.332227 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:39.332184    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nqcrm" podStartSLOduration=4.034493831 podStartE2EDuration="23.332163875s" totalImagesPullingTime="19.297670044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:38.445978693 +0000 UTC m=+23.885475624" watchObservedRunningTime="2026-07-18 01:48:39.332163875 +0000 UTC m=+24.771660807"
Jul 18 01:48:39.332696 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:39.332651    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8wrwt" podStartSLOduration=4.033246104 podStartE2EDuration="23.332641371s" totalImagesPullingTime="19.299395267s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:39.331984017 +0000 UTC m=+24.771480951" watchObservedRunningTime="2026-07-18 01:48:39.332641371 +0000 UTC m=+24.772138303"
Jul 18 01:48:39.457873 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:39.457825    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:39.458401 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:39.458381    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:40.015828 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:40.015710    2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T01:48:39.153383991Z","UUID":"030187c9-625d-4043-9256-0b4266e1b305","Handler":null,"Name":"","Endpoint":""}
Jul 18 01:48:40.019776 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:40.019054    2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 18 01:48:40.019776 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:40.019098    2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 18 01:48:40.326175 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:40.326089    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:40.326783 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:40.326767    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dmwh7"
Jul 18 01:48:41.072944 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:41.072879    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:41.073196 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:41.072983    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:41.073196 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:41.072992    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:41.073196 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:41.073011    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:41.073196 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:41.073104    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:41.073431 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:41.073199    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:41.331035 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:41.330950    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz" event={"ID":"7518d4ee-c387-40a6-afce-250816fdd272","Type":"ContainerStarted","Data":"0b9d5033720efd8271b081eef451a39842ad84dc1f911efe0f8eef87d1f6d8a6"}
Jul 18 01:48:41.334815 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:41.334781    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh" event={"ID":"00845976-489b-41e9-b40b-b4ee2d16a0b3","Type":"ContainerStarted","Data":"d72cad9f42910c2aec07033c4d98074e5a9869ecdbb7afc4a3b66dff339f3064"}
Jul 18 01:48:41.348416 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:41.348365    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvrsz" podStartSLOduration=3.645422061 podStartE2EDuration="25.348349451s" totalImagesPullingTime="21.70292739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:41.348279027 +0000 UTC m=+26.787775979" watchObservedRunningTime="2026-07-18 01:48:41.348349451 +0000 UTC m=+26.787846383"
Jul 18 01:48:43.070234 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:43.069986    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:43.070234 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:43.070091    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:43.071113 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:43.070274    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:43.071113 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:43.070139    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:43.071113 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:43.070339    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:43.071113 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:43.070414    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:43.341064 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:43.340981    2552 generic.go:356] "Generic (PLEG): container finished" podID="422f305c-59b7-41b0-91bd-1fe60a29acda" containerID="d093d02a6cf5363064bab2207df54387635cdc305a247ae94c7e359cb8aea79e" exitCode=0
Jul 18 01:48:43.341226 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:43.341070    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p4kc" event={"ID":"422f305c-59b7-41b0-91bd-1fe60a29acda","Type":"ContainerDied","Data":"d093d02a6cf5363064bab2207df54387635cdc305a247ae94c7e359cb8aea79e"}
Jul 18 01:48:43.344798 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:43.344777    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh" event={"ID":"00845976-489b-41e9-b40b-b4ee2d16a0b3","Type":"ContainerStarted","Data":"d1a0e31427e530e613ee019fdbff5a4b3def976d5b46c76580ca644743a98e40"}
Jul 18 01:48:43.345019 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:43.344998    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:43.345084 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:43.345074    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:43.360799 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:43.360777    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:43.405881 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:43.405830    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh" podStartSLOduration=9.029478869 podStartE2EDuration="28.405814963s" totalImagesPullingTime="19.376336094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:43.405395337 +0000 UTC m=+28.844892269" watchObservedRunningTime="2026-07-18 01:48:43.405814963 +0000 UTC m=+28.845311895"
Jul 18 01:48:44.347266 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:44.347048    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:44.364885 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:44.364668    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:48:44.603065 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:44.602939    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-42wx4"]
Jul 18 01:48:44.603203 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:44.603115    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:44.603241 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:44.603219    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:44.603618 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:44.603600    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fgqqf"]
Jul 18 01:48:44.603696 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:44.603687    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:44.603787 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:44.603771    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:44.604845 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:44.604824    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zr5mn"]
Jul 18 01:48:44.604969 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:44.604917    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:44.605027 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:44.604996    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:45.350869 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:45.350838    2552 generic.go:356] "Generic (PLEG): container finished" podID="422f305c-59b7-41b0-91bd-1fe60a29acda" containerID="a60cf85d42112160ecfbd027623aa92c3ca6afd0795bcc4524a08701b6ce4e99" exitCode=0
Jul 18 01:48:45.351355 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:45.350929    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p4kc" event={"ID":"422f305c-59b7-41b0-91bd-1fe60a29acda","Type":"ContainerDied","Data":"a60cf85d42112160ecfbd027623aa92c3ca6afd0795bcc4524a08701b6ce4e99"}
Jul 18 01:48:46.069593 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:46.069559    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:46.069789 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:46.069612    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:46.069789 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:46.069702    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:46.069789 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:46.069710    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:46.069897 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:46.069796    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:46.069897 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:46.069868    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:47.358151 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:47.358121    2552 generic.go:356] "Generic (PLEG): container finished" podID="422f305c-59b7-41b0-91bd-1fe60a29acda" containerID="92c2884c8529b7577a7e6d0c50972487b76283100f12f4e3f87f574595e66838" exitCode=0
Jul 18 01:48:47.358536 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:47.358184    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p4kc" event={"ID":"422f305c-59b7-41b0-91bd-1fe60a29acda","Type":"ContainerDied","Data":"92c2884c8529b7577a7e6d0c50972487b76283100f12f4e3f87f574595e66838"}
Jul 18 01:48:48.069844 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:48.069810    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:48.070044 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:48.069816    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:48.070044 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:48.069937    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:48.070044 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:48.069936    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:48.070044 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:48.070036    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:48.070238 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:48.070112    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:48.681893 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:48.681848    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:48.682319 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:48.681904    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxq6c\" (UniqueName: \"kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c\") pod \"network-check-target-42wx4\" (UID: \"bb0b5223-b563-4075-8406-c096402a16c5\") " pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:48.682319 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:48.682016    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:48:48.682319 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:48.682016    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:48.682319 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:48.682034    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:48:48.682319 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:48.682049    2552 projected.go:196] Error preparing data for projected volume kube-api-access-cxq6c for pod openshift-network-diagnostics/network-check-target-42wx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:48.682319 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:48.682093    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs podName:4de548f6-694f-425d-9ee4-5ac3df26aed4 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:20.682076113 +0000 UTC m=+66.121573024 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs") pod "network-metrics-daemon-fgqqf" (UID: "4de548f6-694f-425d-9ee4-5ac3df26aed4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:48:48.682319 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:48.682109    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c podName:bb0b5223-b563-4075-8406-c096402a16c5 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:20.682101845 +0000 UTC m=+66.121598754 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxq6c" (UniqueName: "kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c") pod "network-check-target-42wx4" (UID: "bb0b5223-b563-4075-8406-c096402a16c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:48:48.783174 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:48.783135    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:48.783367 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:48.783295    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:48.783367 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:48.783361    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret podName:45860c98-3146-470d-8c2c-3feb65e32e10 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:20.783344654 +0000 UTC m=+66.222841566 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret") pod "global-pull-secret-syncer-zr5mn" (UID: "45860c98-3146-470d-8c2c-3feb65e32e10") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:48:50.070144 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.069632    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:50.070144 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.069668    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:50.070144 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.069668    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:50.070144 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.069776    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgqqf" podUID="4de548f6-694f-425d-9ee4-5ac3df26aed4"
Jul 18 01:48:50.070144 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.069829    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42wx4" podUID="bb0b5223-b563-4075-8406-c096402a16c5"
Jul 18 01:48:50.070144 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.069947    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zr5mn" podUID="45860c98-3146-470d-8c2c-3feb65e32e10"
Jul 18 01:48:50.341842 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.341606    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-48.ec2.internal" event="NodeReady"
Jul 18 01:48:50.341976 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.341914    2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 18 01:48:50.375168 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.375141    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qklh8"]
Jul 18 01:48:50.379582 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.379019    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-99ff6f44d-sngjf"]
Jul 18 01:48:50.379582 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.379191    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qklh8"
Jul 18 01:48:50.383629 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.383586    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gbgdt\""
Jul 18 01:48:50.383757 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.383676    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:48:50.383757 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.383685    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 18 01:48:50.384032 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.384012    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"]
Jul 18 01:48:50.385593 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.385569    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.386834 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.386813    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"]
Jul 18 01:48:50.386980 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.386960    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:48:50.388004 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.387963    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 18 01:48:50.388120 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.388094    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 18 01:48:50.388367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.388335    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tsvv8\""
Jul 18 01:48:50.388618 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.388603    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 18 01:48:50.389259 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.389241    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qklh8"]
Jul 18 01:48:50.390176 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.389316    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"
Jul 18 01:48:50.390176 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.389557    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ml9cq\""
Jul 18 01:48:50.390176 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.389773    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:48:50.390176 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.389973    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 18 01:48:50.390176 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.390090    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 18 01:48:50.391625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.391603    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:48:50.391807 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.391791    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 18 01:48:50.392135 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.391887    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lqm5r\""
Jul 18 01:48:50.392135 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.392032    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 18 01:48:50.393024 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.393004    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 18 01:48:50.394755 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.394736    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 18 01:48:50.403903 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.403878    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"]
Jul 18 01:48:50.404692 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.404631    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"]
Jul 18 01:48:50.405402 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.405379    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"]
Jul 18 01:48:50.407343 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.407324    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-gmrn7"]
Jul 18 01:48:50.407574 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.407556    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:50.409073 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.409046    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dvgh9"]
Jul 18 01:48:50.409922 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.409901    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 18 01:48:50.410017 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.409934    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.410349 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.410332    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 18 01:48:50.411110 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.411090    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-hqv46"]
Jul 18 01:48:50.411209 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.411174    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 18 01:48:50.411382 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.411365    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 18 01:48:50.412277 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.411597    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-95t94\""
Jul 18 01:48:50.412277 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.411888    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.412657 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.412640    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6c89c86f68-22ccf"]
Jul 18 01:48:50.413114 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.412814    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.413114 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.413009    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.413633    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.413838    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.414063    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.414964    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.415241    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s6gqx\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.415477    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.415507    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-w5njt\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.415809    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.417665    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.418310    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"]
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.419541    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qmdmc\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.421266    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.421338    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 18 01:48:50.423260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.422128    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.424688 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.424668    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-99ff6f44d-sngjf"]
Jul 18 01:48:50.425829 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.425810    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"]
Jul 18 01:48:50.426183 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.425255    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"
Jul 18 01:48:50.429451 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.428152    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 18 01:48:50.429451 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.428280    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 18 01:48:50.429451 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.428601    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mfpwx\""
Jul 18 01:48:50.429451 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.428891    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 18 01:48:50.429451 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.429000    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 18 01:48:50.429451 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.429211    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 18 01:48:50.429451 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.429216    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 18 01:48:50.430382 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.430364    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 18 01:48:50.433566 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.433545    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 18 01:48:50.434542 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.434524    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-srh28"]
Jul 18 01:48:50.435187 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.435164    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 18 01:48:50.435283 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.435215    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 18 01:48:50.435409 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.435394    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 18 01:48:50.435498 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.435407    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-rk5pm\""
Jul 18 01:48:50.435665 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.435645    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:48:50.435728 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.435706    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:48:50.436442 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.436409    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bdbxm"]
Jul 18 01:48:50.437360 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.437339    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-srh28"
Jul 18 01:48:50.437828 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.437809    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 18 01:48:50.437900 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.437885    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xnmph\""
Jul 18 01:48:50.437971 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.437816    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 18 01:48:50.438323 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.438307    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"]
Jul 18 01:48:50.438404 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.438358    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-gmrn7"]
Jul 18 01:48:50.438404 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.438382    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dvgh9"]
Jul 18 01:48:50.438539 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.438442    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bdbxm"
Jul 18 01:48:50.438606 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.438444    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"]
Jul 18 01:48:50.438669 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.438652    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-hqv46"]
Jul 18 01:48:50.438725 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.438681    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bdbxm"]
Jul 18 01:48:50.439487 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.439403    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 01:48:50.439487 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.439484    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 01:48:50.439816 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.439800    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r4xfd\""
Jul 18 01:48:50.440739 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.440722    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 18 01:48:50.440865 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.440830    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 18 01:48:50.440865 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.440864    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4j2w9\""
Jul 18 01:48:50.442254 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.442235    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 18 01:48:50.443921 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.443533    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c89c86f68-22ccf"]
Jul 18 01:48:50.445665 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.445643    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-srh28"]
Jul 18 01:48:50.446680 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.446658    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"]
Jul 18 01:48:50.499276 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499253    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-bound-sa-token\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.499405 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499281    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7znq2\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-kube-api-access-7znq2\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.499405 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499297    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2cbad028-6f01-4a6c-941f-966af50c85c0-config\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.499405 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499314    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.499405 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499335    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7081f4df-c702-41ab-b078-d62252a5b207-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:48:50.499405 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499382    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ls5j\" (UniqueName: \"kubernetes.io/projected/d06d5fa3-648b-4092-bbe9-09c8c0176e1b-kube-api-access-4ls5j\") pod \"service-ca-operator-5bcbc94c78-4gjbp\" (UID: \"d06d5fa3-648b-4092-bbe9-09c8c0176e1b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"
Jul 18 01:48:50.499690 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499413    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.499690 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499482    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-image-registry-private-configuration\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.499690 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499521    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-certificates\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.499690 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499563    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.499690 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499589    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxrf5\" (UniqueName: \"kubernetes.io/projected/2cbad028-6f01-4a6c-941f-966af50c85c0-kube-api-access-kxrf5\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.499690 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499615    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djkdb\" (UniqueName: \"kubernetes.io/projected/874287f6-35db-4ecb-9213-64ef5ed15dfa-kube-api-access-djkdb\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.499690 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499642    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31bfb9b1-86b3-4194-8651-68f48a251535-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-7w6l2\" (UID: \"31bfb9b1-86b3-4194-8651-68f48a251535\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499692    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ae958a97-679e-4c2b-bb4d-6c8719563be1-ca-trust-extracted\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499751    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-trusted-ca\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499778    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/874287f6-35db-4ecb-9213-64ef5ed15dfa-config-volume\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499803    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d06d5fa3-648b-4092-bbe9-09c8c0176e1b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-4gjbp\" (UID: \"d06d5fa3-648b-4092-bbe9-09c8c0176e1b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499829    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499853    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e601610-f330-493b-adf7-13534056efa2-cert\") pod \"ingress-canary-bdbxm\" (UID: \"1e601610-f330-493b-adf7-13534056efa2\") " pod="openshift-ingress-canary/ingress-canary-bdbxm"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499877    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-installation-pull-secrets\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499908    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxd7q\" (UniqueName: \"kubernetes.io/projected/1d553bb2-7d34-4365-98b0-7b2b90227719-kube-api-access-fxd7q\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499930    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmww8\" (UniqueName: \"kubernetes.io/projected/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-kube-api-access-wmww8\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499949    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d06d5fa3-648b-4092-bbe9-09c8c0176e1b-config\") pod \"service-ca-operator-5bcbc94c78-4gjbp\" (UID: \"d06d5fa3-648b-4092-bbe9-09c8c0176e1b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.499976    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qrs6\" (UniqueName: \"kubernetes.io/projected/31bfb9b1-86b3-4194-8651-68f48a251535-kube-api-access-4qrs6\") pod \"kube-storage-version-migrator-operator-7658d9b749-7w6l2\" (UID: \"31bfb9b1-86b3-4194-8651-68f48a251535\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500000    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/46a417f1-eed7-4643-a01a-0cb5029dcc78-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:50.500031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500017    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d553bb2-7d34-4365-98b0-7b2b90227719-tmp\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500047    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r6z8\" (UniqueName: \"kubernetes.io/projected/46a417f1-eed7-4643-a01a-0cb5029dcc78-kube-api-access-5r6z8\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500110    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-default-certificate\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500128    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500146    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d553bb2-7d34-4365-98b0-7b2b90227719-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500161    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tq7h\" (UniqueName: \"kubernetes.io/projected/1e601610-f330-493b-adf7-13534056efa2-kube-api-access-9tq7h\") pod \"ingress-canary-bdbxm\" (UID: \"1e601610-f330-493b-adf7-13534056efa2\") " pod="openshift-ingress-canary/ingress-canary-bdbxm"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500192    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d553bb2-7d34-4365-98b0-7b2b90227719-service-ca-bundle\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500219    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500235    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-778n6\" (UniqueName: \"kubernetes.io/projected/24d78e74-d41d-4b85-a543-922c74ee852a-kube-api-access-778n6\") pod \"volume-data-source-validator-78cf4c6f6-qklh8\" (UID: \"24d78e74-d41d-4b85-a543-922c74ee852a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qklh8"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500266    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d553bb2-7d34-4365-98b0-7b2b90227719-serving-cert\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500285    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31bfb9b1-86b3-4194-8651-68f48a251535-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-7w6l2\" (UID: \"31bfb9b1-86b3-4194-8651-68f48a251535\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500300    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500327    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2cbad028-6f01-4a6c-941f-966af50c85c0-serving-cert\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500353    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1d553bb2-7d34-4365-98b0-7b2b90227719-snapshots\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500376    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-stats-auth\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.500663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500391    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2cbad028-6f01-4a6c-941f-966af50c85c0-trusted-ca\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.501338 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500412    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvgw4\" (UniqueName: \"kubernetes.io/projected/4f6d3fbd-c800-40fa-821e-ec6e62ae3b28-kube-api-access-xvgw4\") pod \"network-check-source-7bc5b6bf85-srh28\" (UID: \"4f6d3fbd-c800-40fa-821e-ec6e62ae3b28\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-srh28"
Jul 18 01:48:50.501338 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500450    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/874287f6-35db-4ecb-9213-64ef5ed15dfa-tmp-dir\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.501338 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.500466    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh5vl\" (UniqueName: \"kubernetes.io/projected/93ed0884-1025-4a1c-b573-dfa09a741a37-kube-api-access-gh5vl\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.600991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.600909    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31bfb9b1-86b3-4194-8651-68f48a251535-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-7w6l2\" (UID: \"31bfb9b1-86b3-4194-8651-68f48a251535\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"
Jul 18 01:48:50.600991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.600948    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ae958a97-679e-4c2b-bb4d-6c8719563be1-ca-trust-extracted\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.600991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.600988    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-trusted-ca\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.601361    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/874287f6-35db-4ecb-9213-64ef5ed15dfa-config-volume\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.601474    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d06d5fa3-648b-4092-bbe9-09c8c0176e1b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-4gjbp\" (UID: \"d06d5fa3-648b-4092-bbe9-09c8c0176e1b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.601512    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.601667    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.601767    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert podName:7081f4df-c702-41ab-b078-d62252a5b207 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:51.101745 +0000 UTC m=+36.541242004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hzgdx" (UID: "7081f4df-c702-41ab-b078-d62252a5b207") : secret "networking-console-plugin-cert" not found
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.601816    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ae958a97-679e-4c2b-bb4d-6c8719563be1-ca-trust-extracted\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.601881    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e601610-f330-493b-adf7-13534056efa2-cert\") pod \"ingress-canary-bdbxm\" (UID: \"1e601610-f330-493b-adf7-13534056efa2\") " pod="openshift-ingress-canary/ingress-canary-bdbxm"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.601917    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-installation-pull-secrets\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.601949    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxd7q\" (UniqueName: \"kubernetes.io/projected/1d553bb2-7d34-4365-98b0-7b2b90227719-kube-api-access-fxd7q\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.601987    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmww8\" (UniqueName: \"kubernetes.io/projected/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-kube-api-access-wmww8\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.602011    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d06d5fa3-648b-4092-bbe9-09c8c0176e1b-config\") pod \"service-ca-operator-5bcbc94c78-4gjbp\" (UID: \"d06d5fa3-648b-4092-bbe9-09c8c0176e1b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.602044    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qrs6\" (UniqueName: \"kubernetes.io/projected/31bfb9b1-86b3-4194-8651-68f48a251535-kube-api-access-4qrs6\") pod \"kube-storage-version-migrator-operator-7658d9b749-7w6l2\" (UID: \"31bfb9b1-86b3-4194-8651-68f48a251535\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.602206    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/874287f6-35db-4ecb-9213-64ef5ed15dfa-config-volume\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.602228    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/46a417f1-eed7-4643-a01a-0cb5029dcc78-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:50.602339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.602301    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31bfb9b1-86b3-4194-8651-68f48a251535-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-7w6l2\" (UID: \"31bfb9b1-86b3-4194-8651-68f48a251535\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.602848    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d553bb2-7d34-4365-98b0-7b2b90227719-tmp\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.602897    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d06d5fa3-648b-4092-bbe9-09c8c0176e1b-config\") pod \"service-ca-operator-5bcbc94c78-4gjbp\" (UID: \"d06d5fa3-648b-4092-bbe9-09c8c0176e1b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.602916    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r6z8\" (UniqueName: \"kubernetes.io/projected/46a417f1-eed7-4643-a01a-0cb5029dcc78-kube-api-access-5r6z8\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.602950    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-default-certificate\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.602968    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-trusted-ca\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.602981    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603032    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d553bb2-7d34-4365-98b0-7b2b90227719-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603070    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tq7h\" (UniqueName: \"kubernetes.io/projected/1e601610-f330-493b-adf7-13534056efa2-kube-api-access-9tq7h\") pod \"ingress-canary-bdbxm\" (UID: \"1e601610-f330-493b-adf7-13534056efa2\") " pod="openshift-ingress-canary/ingress-canary-bdbxm"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.603073    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603108    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d553bb2-7d34-4365-98b0-7b2b90227719-service-ca-bundle\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.603136    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls podName:b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab nodeName:}" failed. No retries permitted until 2026-07-18 01:48:51.103112376 +0000 UTC m=+36.542609300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-d6m27" (UID: "b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab") : secret "samples-operator-tls" not found
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603177    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603203    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d553bb2-7d34-4365-98b0-7b2b90227719-tmp\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603108    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/46a417f1-eed7-4643-a01a-0cb5029dcc78-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:50.603213 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603205    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-778n6\" (UniqueName: \"kubernetes.io/projected/24d78e74-d41d-4b85-a543-922c74ee852a-kube-api-access-778n6\") pod \"volume-data-source-validator-78cf4c6f6-qklh8\" (UID: \"24d78e74-d41d-4b85-a543-922c74ee852a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qklh8"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603261    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d553bb2-7d34-4365-98b0-7b2b90227719-serving-cert\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603295    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31bfb9b1-86b3-4194-8651-68f48a251535-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-7w6l2\" (UID: \"31bfb9b1-86b3-4194-8651-68f48a251535\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603327    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603365    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2cbad028-6f01-4a6c-941f-966af50c85c0-serving-cert\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603391    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1d553bb2-7d34-4365-98b0-7b2b90227719-snapshots\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603474    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-stats-auth\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603503    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2cbad028-6f01-4a6c-941f-966af50c85c0-trusted-ca\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603531    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvgw4\" (UniqueName: \"kubernetes.io/projected/4f6d3fbd-c800-40fa-821e-ec6e62ae3b28-kube-api-access-xvgw4\") pod \"network-check-source-7bc5b6bf85-srh28\" (UID: \"4f6d3fbd-c800-40fa-821e-ec6e62ae3b28\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-srh28"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603561    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/874287f6-35db-4ecb-9213-64ef5ed15dfa-tmp-dir\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603584    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh5vl\" (UniqueName: \"kubernetes.io/projected/93ed0884-1025-4a1c-b573-dfa09a741a37-kube-api-access-gh5vl\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603621    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-bound-sa-token\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603653    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7znq2\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-kube-api-access-7znq2\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.603991 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.603719    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d553bb2-7d34-4365-98b0-7b2b90227719-service-ca-bundle\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.604639 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.604047    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:48:50.604639 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.604118    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls podName:46a417f1-eed7-4643-a01a-0cb5029dcc78 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:51.104099454 +0000 UTC m=+36.543596365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-m5m8z" (UID: "46a417f1-eed7-4643-a01a-0cb5029dcc78") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:48:50.605264 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.605014    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d553bb2-7d34-4365-98b0-7b2b90227719-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.605264 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.605119    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:48:50.605264 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.605181    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:51.105161018 +0000 UTC m=+36.544657942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : secret "router-metrics-certs-default" not found
Jul 18 01:48:50.605680 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.605573    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2cbad028-6f01-4a6c-941f-966af50c85c0-config\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.605680 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.605611    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.605680 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.605646    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7081f4df-c702-41ab-b078-d62252a5b207-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:48:50.605680 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.605675    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ls5j\" (UniqueName: \"kubernetes.io/projected/d06d5fa3-648b-4092-bbe9-09c8c0176e1b-kube-api-access-4ls5j\") pod \"service-ca-operator-5bcbc94c78-4gjbp\" (UID: \"d06d5fa3-648b-4092-bbe9-09c8c0176e1b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"
Jul 18 01:48:50.606024 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.605706    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.606024 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.605751    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-image-registry-private-configuration\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.606024 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.605784    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-certificates\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.606024 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.605832    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.606024 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.605865    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxrf5\" (UniqueName: \"kubernetes.io/projected/2cbad028-6f01-4a6c-941f-966af50c85c0-kube-api-access-kxrf5\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.606024 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.605896    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djkdb\" (UniqueName: \"kubernetes.io/projected/874287f6-35db-4ecb-9213-64ef5ed15dfa-kube-api-access-djkdb\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.606969 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.606938    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2cbad028-6f01-4a6c-941f-966af50c85c0-config\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.607090 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.607070    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:48:50.607149 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.607132    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls podName:874287f6-35db-4ecb-9213-64ef5ed15dfa nodeName:}" failed. No retries permitted until 2026-07-18 01:48:51.107117054 +0000 UTC m=+36.546613966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls") pod "dns-default-dvgh9" (UID: "874287f6-35db-4ecb-9213-64ef5ed15dfa") : secret "dns-default-metrics-tls" not found
Jul 18 01:48:50.607793 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.607770    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7081f4df-c702-41ab-b078-d62252a5b207-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:48:50.608131 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.608105    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:48:50.608220 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.608135    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-99ff6f44d-sngjf: secret "image-registry-tls" not found
Jul 18 01:48:50.608220 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.608180    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls podName:ae958a97-679e-4c2b-bb4d-6c8719563be1 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:51.108160403 +0000 UTC m=+36.547657329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls") pod "image-registry-99ff6f44d-sngjf" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1") : secret "image-registry-tls" not found
Jul 18 01:48:50.608337 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.608223    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2cbad028-6f01-4a6c-941f-966af50c85c0-trusted-ca\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.608732 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.608708    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-stats-auth\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.608834 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.608760    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/874287f6-35db-4ecb-9213-64ef5ed15dfa-tmp-dir\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.609102 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.609084    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-installation-pull-secrets\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.609302 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:50.609286    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:51.109106852 +0000 UTC m=+36.548603767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : configmap references non-existent config key: service-ca.crt
Jul 18 01:48:50.609567 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.609545    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-default-certificate\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.609791 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.608111    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1d553bb2-7d34-4365-98b0-7b2b90227719-snapshots\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.609866 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.609800    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-certificates\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.611090 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.611067    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-image-registry-private-configuration\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.613556 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.613532    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d553bb2-7d34-4365-98b0-7b2b90227719-serving-cert\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.614564 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.614540    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djkdb\" (UniqueName: \"kubernetes.io/projected/874287f6-35db-4ecb-9213-64ef5ed15dfa-kube-api-access-djkdb\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:50.615216 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.615162    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d06d5fa3-648b-4092-bbe9-09c8c0176e1b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-4gjbp\" (UID: \"d06d5fa3-648b-4092-bbe9-09c8c0176e1b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"
Jul 18 01:48:50.616062 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.615510    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e601610-f330-493b-adf7-13534056efa2-cert\") pod \"ingress-canary-bdbxm\" (UID: \"1e601610-f330-493b-adf7-13534056efa2\") " pod="openshift-ingress-canary/ingress-canary-bdbxm"
Jul 18 01:48:50.616062 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.615575    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31bfb9b1-86b3-4194-8651-68f48a251535-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-7w6l2\" (UID: \"31bfb9b1-86b3-4194-8651-68f48a251535\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"
Jul 18 01:48:50.616062 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.615837    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2cbad028-6f01-4a6c-941f-966af50c85c0-serving-cert\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.616062 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.615934    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmww8\" (UniqueName: \"kubernetes.io/projected/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-kube-api-access-wmww8\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:48:50.616330 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.616103    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxd7q\" (UniqueName: \"kubernetes.io/projected/1d553bb2-7d34-4365-98b0-7b2b90227719-kube-api-access-fxd7q\") pod \"insights-operator-555d9d7885-hqv46\" (UID: \"1d553bb2-7d34-4365-98b0-7b2b90227719\") " pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.616559 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.616537    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ls5j\" (UniqueName: \"kubernetes.io/projected/d06d5fa3-648b-4092-bbe9-09c8c0176e1b-kube-api-access-4ls5j\") pod \"service-ca-operator-5bcbc94c78-4gjbp\" (UID: \"d06d5fa3-648b-4092-bbe9-09c8c0176e1b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"
Jul 18 01:48:50.617010 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.616986    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qrs6\" (UniqueName: \"kubernetes.io/projected/31bfb9b1-86b3-4194-8651-68f48a251535-kube-api-access-4qrs6\") pod \"kube-storage-version-migrator-operator-7658d9b749-7w6l2\" (UID: \"31bfb9b1-86b3-4194-8651-68f48a251535\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"
Jul 18 01:48:50.617393 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.617374    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r6z8\" (UniqueName: \"kubernetes.io/projected/46a417f1-eed7-4643-a01a-0cb5029dcc78-kube-api-access-5r6z8\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:50.618079 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.618056    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-778n6\" (UniqueName: \"kubernetes.io/projected/24d78e74-d41d-4b85-a543-922c74ee852a-kube-api-access-778n6\") pod \"volume-data-source-validator-78cf4c6f6-qklh8\" (UID: \"24d78e74-d41d-4b85-a543-922c74ee852a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qklh8"
Jul 18 01:48:50.619381 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.619361    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxrf5\" (UniqueName: \"kubernetes.io/projected/2cbad028-6f01-4a6c-941f-966af50c85c0-kube-api-access-kxrf5\") pod \"console-operator-565b7b569d-gmrn7\" (UID: \"2cbad028-6f01-4a6c-941f-966af50c85c0\") " pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.619514 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.619492    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvgw4\" (UniqueName: \"kubernetes.io/projected/4f6d3fbd-c800-40fa-821e-ec6e62ae3b28-kube-api-access-xvgw4\") pod \"network-check-source-7bc5b6bf85-srh28\" (UID: \"4f6d3fbd-c800-40fa-821e-ec6e62ae3b28\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-srh28"
Jul 18 01:48:50.619900 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.619881    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh5vl\" (UniqueName: \"kubernetes.io/projected/93ed0884-1025-4a1c-b573-dfa09a741a37-kube-api-access-gh5vl\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:50.621067 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.621047    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7znq2\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-kube-api-access-7znq2\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.621163 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.621049    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tq7h\" (UniqueName: \"kubernetes.io/projected/1e601610-f330-493b-adf7-13534056efa2-kube-api-access-9tq7h\") pod \"ingress-canary-bdbxm\" (UID: \"1e601610-f330-493b-adf7-13534056efa2\") " pod="openshift-ingress-canary/ingress-canary-bdbxm"
Jul 18 01:48:50.621277 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.621258    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-bound-sa-token\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:50.697129 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.697100    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qklh8"
Jul 18 01:48:50.732828 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.732787    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"
Jul 18 01:48:50.749084 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.749048    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:48:50.763125 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.763099    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-hqv46"
Jul 18 01:48:50.793059 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.793018    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"
Jul 18 01:48:50.807626 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.807603    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-srh28"
Jul 18 01:48:50.813789 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:50.813769    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bdbxm"
Jul 18 01:48:51.109726 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.109690    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:51.109726 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.109728    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.109764    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.109820    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.109863    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.109872    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.109893    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-99ff6f44d-sngjf: secret "image-registry-tls" not found
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.109924    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls podName:874287f6-35db-4ecb-9213-64ef5ed15dfa nodeName:}" failed. No retries permitted until 2026-07-18 01:48:52.109907167 +0000 UTC m=+37.549404096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls") pod "dns-default-dvgh9" (UID: "874287f6-35db-4ecb-9213-64ef5ed15dfa") : secret "dns-default-metrics-tls" not found
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.109872    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.109942    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls podName:ae958a97-679e-4c2b-bb4d-6c8719563be1 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:52.109933418 +0000 UTC m=+37.549430327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls") pod "image-registry-99ff6f44d-sngjf" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1") : secret "image-registry-tls" not found
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.109992    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:52.109975997 +0000 UTC m=+37.549472907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : configmap references non-existent config key: service-ca.crt
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.109995    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.110016    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.110034    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert podName:7081f4df-c702-41ab-b078-d62252a5b207 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:52.110020271 +0000 UTC m=+37.549517181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hzgdx" (UID: "7081f4df-c702-41ab-b078-d62252a5b207") : secret "networking-console-plugin-cert" not found
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.110061    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.110088    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:48:51.110455 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.110122    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls podName:46a417f1-eed7-4643-a01a-0cb5029dcc78 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:52.110110654 +0000 UTC m=+37.549607571 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-m5m8z" (UID: "46a417f1-eed7-4643-a01a-0cb5029dcc78") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:48:51.110981 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.109946    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:48:51.110981 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.110152    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:48:51.110981 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.110158    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls podName:b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab nodeName:}" failed. No retries permitted until 2026-07-18 01:48:52.110147586 +0000 UTC m=+37.549644525 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-d6m27" (UID: "b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab") : secret "samples-operator-tls" not found
Jul 18 01:48:51.110981 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:51.110177    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:52.110169354 +0000 UTC m=+37.549666263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : secret "router-metrics-certs-default" not found
Jul 18 01:48:51.676171 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.675753    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bdbxm"]
Jul 18 01:48:51.683216 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:51.683182    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e601610_f330_493b_adf7_13534056efa2.slice/crio-9470a5d680fe470cecb1653103baeef81dd7e7c12d5bb957b9d6b2de23a9ab0a WatchSource:0}: Error finding container 9470a5d680fe470cecb1653103baeef81dd7e7c12d5bb957b9d6b2de23a9ab0a: Status 404 returned error can't find the container with id 9470a5d680fe470cecb1653103baeef81dd7e7c12d5bb957b9d6b2de23a9ab0a
Jul 18 01:48:51.703393 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.703367    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp"]
Jul 18 01:48:51.727411 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.727315    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qklh8"]
Jul 18 01:48:51.731751 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.730695    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-gmrn7"]
Jul 18 01:48:51.732062 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.732003    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2"]
Jul 18 01:48:51.732961 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.732940    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-srh28"]
Jul 18 01:48:51.744416 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:51.744381    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-hqv46"]
Jul 18 01:48:51.747341 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:51.747313    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d78e74_d41d_4b85_a543_922c74ee852a.slice/crio-e85156c2bc5f4fe783a5fea6b3f79e311ec4ba16c3b4b780ecfee6e723ee63e8 WatchSource:0}: Error finding container e85156c2bc5f4fe783a5fea6b3f79e311ec4ba16c3b4b780ecfee6e723ee63e8: Status 404 returned error can't find the container with id e85156c2bc5f4fe783a5fea6b3f79e311ec4ba16c3b4b780ecfee6e723ee63e8
Jul 18 01:48:51.748542 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:51.748227    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d553bb2_7d34_4365_98b0_7b2b90227719.slice/crio-b6e01be33b855bfe26fd26e60d0a470254c68c8eda62c8c5e5ec1ebbd1740336 WatchSource:0}: Error finding container b6e01be33b855bfe26fd26e60d0a470254c68c8eda62c8c5e5ec1ebbd1740336: Status 404 returned error can't find the container with id b6e01be33b855bfe26fd26e60d0a470254c68c8eda62c8c5e5ec1ebbd1740336
Jul 18 01:48:51.748986 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:51.748967    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cbad028_6f01_4a6c_941f_966af50c85c0.slice/crio-97d33e0558cb4fcb605a53c6aa249ac3f9904cb1b3ce301b8260be8737172991 WatchSource:0}: Error finding container 97d33e0558cb4fcb605a53c6aa249ac3f9904cb1b3ce301b8260be8737172991: Status 404 returned error can't find the container with id 97d33e0558cb4fcb605a53c6aa249ac3f9904cb1b3ce301b8260be8737172991
Jul 18 01:48:51.752761 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:51.752046    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31bfb9b1_86b3_4194_8651_68f48a251535.slice/crio-4e5370e83597cce111c22ab4b438297af18c65d43557c63fb872c46acf2d203e WatchSource:0}: Error finding container 4e5370e83597cce111c22ab4b438297af18c65d43557c63fb872c46acf2d203e: Status 404 returned error can't find the container with id 4e5370e83597cce111c22ab4b438297af18c65d43557c63fb872c46acf2d203e
Jul 18 01:48:51.753161 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:48:51.752970    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06d5fa3_648b_4092_bbe9_09c8c0176e1b.slice/crio-4728be853f1639c48834313ed25e6af66a73b943a3383dc130b64ac17651e550 WatchSource:0}: Error finding container 4728be853f1639c48834313ed25e6af66a73b943a3383dc130b64ac17651e550: Status 404 returned error can't find the container with id 4728be853f1639c48834313ed25e6af66a73b943a3383dc130b64ac17651e550
Jul 18 01:48:52.069916 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.069832    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:48:52.069916 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.069896    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:48:52.070104 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.069994    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:48:52.072513 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.072493    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mcksx\""
Jul 18 01:48:52.072639 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.072544    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 01:48:52.072703 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.072690    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 01:48:52.072884 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.072866    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2rqj2\""
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.122551    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.122601    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.122658    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.122690    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.122738    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.122786    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.122845    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.122980    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123040    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls podName:b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.123021547 +0000 UTC m=+39.562518464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-d6m27" (UID: "b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab") : secret "samples-operator-tls" not found
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123480    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123531    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls podName:46a417f1-eed7-4643-a01a-0cb5029dcc78 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.123516254 +0000 UTC m=+39.563013167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-m5m8z" (UID: "46a417f1-eed7-4643-a01a-0cb5029dcc78") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123592    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123624    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.123613207 +0000 UTC m=+39.563110123 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : secret "router-metrics-certs-default" not found
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123680    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123709    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls podName:874287f6-35db-4ecb-9213-64ef5ed15dfa nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.123699469 +0000 UTC m=+39.563196383 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls") pod "dns-default-dvgh9" (UID: "874287f6-35db-4ecb-9213-64ef5ed15dfa") : secret "dns-default-metrics-tls" not found
Jul 18 01:48:52.123994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123774    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:48:52.125493 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123783    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-99ff6f44d-sngjf: secret "image-registry-tls" not found
Jul 18 01:48:52.125493 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123813    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls podName:ae958a97-679e-4c2b-bb4d-6c8719563be1 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.12380322 +0000 UTC m=+39.563300145 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls") pod "image-registry-99ff6f44d-sngjf" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1") : secret "image-registry-tls" not found
Jul 18 01:48:52.125493 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123880    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.123869791 +0000 UTC m=+39.563366706 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : configmap references non-existent config key: service-ca.crt
Jul 18 01:48:52.125493 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123937    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:48:52.125493 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:52.123967    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert podName:7081f4df-c702-41ab-b078-d62252a5b207 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.123955974 +0000 UTC m=+39.563452889 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hzgdx" (UID: "7081f4df-c702-41ab-b078-d62252a5b207") : secret "networking-console-plugin-cert" not found
Jul 18 01:48:52.378452 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.378334    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp" event={"ID":"d06d5fa3-648b-4092-bbe9-09c8c0176e1b","Type":"ContainerStarted","Data":"4728be853f1639c48834313ed25e6af66a73b943a3383dc130b64ac17651e550"}
Jul 18 01:48:52.381148 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.381119    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-hqv46" event={"ID":"1d553bb2-7d34-4365-98b0-7b2b90227719","Type":"ContainerStarted","Data":"b6e01be33b855bfe26fd26e60d0a470254c68c8eda62c8c5e5ec1ebbd1740336"}
Jul 18 01:48:52.383461 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.383411    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2" event={"ID":"31bfb9b1-86b3-4194-8651-68f48a251535","Type":"ContainerStarted","Data":"4e5370e83597cce111c22ab4b438297af18c65d43557c63fb872c46acf2d203e"}
Jul 18 01:48:52.385668 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.385644    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-srh28" event={"ID":"4f6d3fbd-c800-40fa-821e-ec6e62ae3b28","Type":"ContainerStarted","Data":"d7dfcb26df386f811c67d42ec9fb9bbc807adc4bf75b41cc8a5b3adb4e1f5de4"}
Jul 18 01:48:52.389381 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.389357    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qklh8" event={"ID":"24d78e74-d41d-4b85-a543-922c74ee852a","Type":"ContainerStarted","Data":"e85156c2bc5f4fe783a5fea6b3f79e311ec4ba16c3b4b780ecfee6e723ee63e8"}
Jul 18 01:48:52.393625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.393596    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bdbxm" event={"ID":"1e601610-f330-493b-adf7-13534056efa2","Type":"ContainerStarted","Data":"9470a5d680fe470cecb1653103baeef81dd7e7c12d5bb957b9d6b2de23a9ab0a"}
Jul 18 01:48:52.400267 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.400239    2552 generic.go:356] "Generic (PLEG): container finished" podID="422f305c-59b7-41b0-91bd-1fe60a29acda" containerID="ec711dd2b47f4d23ec26f10d0ed9cc5af18650ad1bf2a37fcc8e0c252a6d6e54" exitCode=0
Jul 18 01:48:52.400392 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.400327    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p4kc" event={"ID":"422f305c-59b7-41b0-91bd-1fe60a29acda","Type":"ContainerDied","Data":"ec711dd2b47f4d23ec26f10d0ed9cc5af18650ad1bf2a37fcc8e0c252a6d6e54"}
Jul 18 01:48:52.405996 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:52.405969    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7" event={"ID":"2cbad028-6f01-4a6c-941f-966af50c85c0","Type":"ContainerStarted","Data":"97d33e0558cb4fcb605a53c6aa249ac3f9904cb1b3ce301b8260be8737172991"}
Jul 18 01:48:53.420638 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:53.420137    2552 generic.go:356] "Generic (PLEG): container finished" podID="422f305c-59b7-41b0-91bd-1fe60a29acda" containerID="d4e9c41ad3212bd267cf6ec57437d2828445b2e3c6ddb82d30b4e934da89c9fe" exitCode=0
Jul 18 01:48:53.420638 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:53.420232    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p4kc" event={"ID":"422f305c-59b7-41b0-91bd-1fe60a29acda","Type":"ContainerDied","Data":"d4e9c41ad3212bd267cf6ec57437d2828445b2e3c6ddb82d30b4e934da89c9fe"}
Jul 18 01:48:54.144147 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.143888    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:48:54.144147 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144135    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:48:54.144352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.144178    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:54.144352 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144212    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls podName:b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab nodeName:}" failed. No retries permitted until 2026-07-18 01:48:58.144191374 +0000 UTC m=+43.583688297 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-d6m27" (UID: "b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab") : secret "samples-operator-tls" not found
Jul 18 01:48:54.144352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.144258    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:54.144352 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144273    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:48:54.144352 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144324    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls podName:46a417f1-eed7-4643-a01a-0cb5029dcc78 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:58.144308293 +0000 UTC m=+43.583805206 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-m5m8z" (UID: "46a417f1-eed7-4643-a01a-0cb5029dcc78") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:48:54.144352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.144343    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:54.144677 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144361    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:48:54.144677 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.144382    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:54.144677 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144397    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:58.144385183 +0000 UTC m=+43.583882097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : secret "router-metrics-certs-default" not found
Jul 18 01:48:54.144677 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144476    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:48:54.144677 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144488    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-99ff6f44d-sngjf: secret "image-registry-tls" not found
Jul 18 01:48:54.144677 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144514    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls podName:ae958a97-679e-4c2b-bb4d-6c8719563be1 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:58.144506408 +0000 UTC m=+43.584003317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls") pod "image-registry-99ff6f44d-sngjf" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1") : secret "image-registry-tls" not found
Jul 18 01:48:54.144677 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.144473    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:54.144677 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.144571    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:48:54.144677 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144657    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:48:54.144677 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144683    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert podName:7081f4df-c702-41ab-b078-d62252a5b207 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:58.1446761 +0000 UTC m=+43.584173009 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hzgdx" (UID: "7081f4df-c702-41ab-b078-d62252a5b207") : secret "networking-console-plugin-cert" not found
Jul 18 01:48:54.145084 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144738    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:48:54.145084 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144769    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls podName:874287f6-35db-4ecb-9213-64ef5ed15dfa nodeName:}" failed. No retries permitted until 2026-07-18 01:48:58.144759105 +0000 UTC m=+43.584256018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls") pod "dns-default-dvgh9" (UID: "874287f6-35db-4ecb-9213-64ef5ed15dfa") : secret "dns-default-metrics-tls" not found
Jul 18 01:48:54.145084 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:54.144787    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:58.144776155 +0000 UTC m=+43.584273083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : configmap references non-existent config key: service-ca.crt
Jul 18 01:48:54.427864 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.427821    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p4kc" event={"ID":"422f305c-59b7-41b0-91bd-1fe60a29acda","Type":"ContainerStarted","Data":"9e98ade7983ea02d99bea7c785bd3b1a3b4ee94f8d2d4fa9e4db3ed943688fec"}
Jul 18 01:48:54.429747 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.429719    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qklh8" event={"ID":"24d78e74-d41d-4b85-a543-922c74ee852a","Type":"ContainerStarted","Data":"b980d1927bac0133144d923dacaa236cc4d6d37449fd728d8f064effc2950859"}
Jul 18 01:48:54.454590 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.453628    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7p4kc" podStartSLOduration=9.629989808 podStartE2EDuration="39.453609068s" totalImagesPullingTime="29.82361926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:54.452246251 +0000 UTC m=+39.891743215" watchObservedRunningTime="2026-07-18 01:48:54.453609068 +0000 UTC m=+39.893106002"
Jul 18 01:48:54.470539 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.470461    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qklh8" podStartSLOduration=14.483161991 podStartE2EDuration="16.47041466s" totalImagesPullingTime="1.987252669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:54.469695089 +0000 UTC m=+39.909192023" watchObservedRunningTime="2026-07-18 01:48:54.47041466 +0000 UTC m=+39.909911589"
Jul 18 01:48:54.585195 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.585155    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:54.670593 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.670530    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:54.737903 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:54.737816    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:55.436291 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:55.436233    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:55.500633 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:55.500594    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:55.920866 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:55.920827    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:55.988874 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:55.988843    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:56.446376 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:56.446331    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:56.510345 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:56.510295    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:48:58.183255 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:58.183222    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:58.183270    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:58.183379    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:58.183462    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183391    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183521    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-99ff6f44d-sngjf: secret "image-registry-tls" not found
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183529    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:06.18351377 +0000 UTC m=+51.623010680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : configmap references non-existent config key: service-ca.crt
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:58.183557    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183571    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls podName:ae958a97-679e-4c2b-bb4d-6c8719563be1 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:06.183555167 +0000 UTC m=+51.623052078 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls") pod "image-registry-99ff6f44d-sngjf" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1") : secret "image-registry-tls" not found
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183579    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183436    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183619    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert podName:7081f4df-c702-41ab-b078-d62252a5b207 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:06.18360941 +0000 UTC m=+51.623106320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hzgdx" (UID: "7081f4df-c702-41ab-b078-d62252a5b207") : secret "networking-console-plugin-cert" not found
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:58.183616    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183638    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183666    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls podName:874287f6-35db-4ecb-9213-64ef5ed15dfa nodeName:}" failed. No retries permitted until 2026-07-18 01:49:06.183641315 +0000 UTC m=+51.623138240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls") pod "dns-default-dvgh9" (UID: "874287f6-35db-4ecb-9213-64ef5ed15dfa") : secret "dns-default-metrics-tls" not found
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:58.183690    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183694    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:48:58.183743 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183751    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:48:58.184655 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183720    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls podName:b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab nodeName:}" failed. No retries permitted until 2026-07-18 01:49:06.18370438 +0000 UTC m=+51.623201295 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-d6m27" (UID: "b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab") : secret "samples-operator-tls" not found
Jul 18 01:48:58.184655 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183777    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls podName:46a417f1-eed7-4643-a01a-0cb5029dcc78 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:06.18376677 +0000 UTC m=+51.623263687 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-m5m8z" (UID: "46a417f1-eed7-4643-a01a-0cb5029dcc78") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:48:58.184655 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:48:58.183802    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:06.183789735 +0000 UTC m=+51.623286648 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : secret "router-metrics-certs-default" not found
Jul 18 01:48:59.449384 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:59.449270    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-hqv46" event={"ID":"1d553bb2-7d34-4365-98b0-7b2b90227719","Type":"ContainerStarted","Data":"73628995b8a151b43568bc2e3ff291306a223eea707a4d1b4be238ee58ad96e4"}
Jul 18 01:48:59.450835 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:59.450800    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2" event={"ID":"31bfb9b1-86b3-4194-8651-68f48a251535","Type":"ContainerStarted","Data":"61e7512a5ce8107e3329d53fc35fe66c646e80f296ecda3edeed78309201b6d2"}
Jul 18 01:48:59.471951 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:59.471874    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2" podStartSLOduration=13.957015215 podStartE2EDuration="21.471854013s" totalImagesPullingTime="7.514838798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:59.471615573 +0000 UTC m=+44.911112506" watchObservedRunningTime="2026-07-18 01:48:59.471854013 +0000 UTC m=+44.911350947"
Jul 18 01:48:59.489768 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:48:59.489701    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-hqv46" podStartSLOduration=13.969951693 podStartE2EDuration="21.489685227s" totalImagesPullingTime="7.519733534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:59.488683554 +0000 UTC m=+44.928180487" watchObservedRunningTime="2026-07-18 01:48:59.489685227 +0000 UTC m=+44.929182210"
Jul 18 01:49:00.247304 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.247258    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k9jpv"]
Jul 18 01:49:00.264043 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.264007    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k9jpv"]
Jul 18 01:49:00.264205 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.264129    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.267517 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.266607    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 18 01:49:00.267517 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.266725    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 18 01:49:00.267517 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.266879    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jz89w\""
Jul 18 01:49:00.294861 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.294816    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:00.407562 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.407519    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77725796-3a90-4975-aaad-61067e30c8a3-data-volume\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.407562 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.407557    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.407814 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.407753    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77725796-3a90-4975-aaad-61067e30c8a3-crio-socket\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.407870 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.407833    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.407920 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.407888    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2sbx\" (UniqueName: \"kubernetes.io/projected/77725796-3a90-4975-aaad-61067e30c8a3-kube-api-access-q2sbx\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.441733 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.441648    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:00.459729 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.459688    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp" event={"ID":"d06d5fa3-648b-4092-bbe9-09c8c0176e1b","Type":"ContainerStarted","Data":"6e440f214527b7728654ed116bd32d6174ce1700645e93fb4c771f5c1f3ee339"}
Jul 18 01:49:00.461241 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.461207    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-srh28" event={"ID":"4f6d3fbd-c800-40fa-821e-ec6e62ae3b28","Type":"ContainerStarted","Data":"a6bf3a6567de6d6c33c0539f7b70e26f003486ba52b405a186a95bf93621a0b2"}
Jul 18 01:49:00.462845 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.462814    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bdbxm" event={"ID":"1e601610-f330-493b-adf7-13534056efa2","Type":"ContainerStarted","Data":"430bf35472b1be29086f38b4e336d9ed18fe59b0cab30426a5801f2c1c699f3f"}
Jul 18 01:49:00.464512 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.464490    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/0.log"
Jul 18 01:49:00.464629 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.464532    2552 generic.go:356] "Generic (PLEG): container finished" podID="2cbad028-6f01-4a6c-941f-966af50c85c0" containerID="90d3cb2c21339086f9ccd1387a638e9ce13fed20e7b0bc7b148e3f87bdb4a09b" exitCode=255
Jul 18 01:49:00.464695 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.464622    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7" event={"ID":"2cbad028-6f01-4a6c-941f-966af50c85c0","Type":"ContainerDied","Data":"90d3cb2c21339086f9ccd1387a638e9ce13fed20e7b0bc7b148e3f87bdb4a09b"}
Jul 18 01:49:00.464833 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.464813    2552 scope.go:117] "RemoveContainer" containerID="90d3cb2c21339086f9ccd1387a638e9ce13fed20e7b0bc7b148e3f87bdb4a09b"
Jul 18 01:49:00.474949 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.474904    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp" podStartSLOduration=14.960898205 podStartE2EDuration="22.47488648s" totalImagesPullingTime="7.513988275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:00.474163529 +0000 UTC m=+45.913660488" watchObservedRunningTime="2026-07-18 01:49:00.47488648 +0000 UTC m=+45.914383409"
Jul 18 01:49:00.490394 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.490347    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-srh28" podStartSLOduration=14.961583167 podStartE2EDuration="22.490310643s" totalImagesPullingTime="7.528727476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:00.489088689 +0000 UTC m=+45.928585635" watchObservedRunningTime="2026-07-18 01:49:00.490310643 +0000 UTC m=+45.929807582"
Jul 18 01:49:00.509287 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.509209    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77725796-3a90-4975-aaad-61067e30c8a3-crio-socket\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.509468 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.509300    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.509468 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.509302    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77725796-3a90-4975-aaad-61067e30c8a3-crio-socket\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.509468 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.509378    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2sbx\" (UniqueName: \"kubernetes.io/projected/77725796-3a90-4975-aaad-61067e30c8a3-kube-api-access-q2sbx\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.509654 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:00.509466    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:49:00.509654 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.509484    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77725796-3a90-4975-aaad-61067e30c8a3-data-volume\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.509654 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.509512    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.509654 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:00.509532    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls podName:77725796-3a90-4975-aaad-61067e30c8a3 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:01.009511181 +0000 UTC m=+46.449008112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9jpv" (UID: "77725796-3a90-4975-aaad-61067e30c8a3") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:49:00.509654 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:00.509563    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:49:00.509654 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:00.509611    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm podName:77725796-3a90-4975-aaad-61067e30c8a3 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:01.009595436 +0000 UTC m=+46.449092353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9jpv" (UID: "77725796-3a90-4975-aaad-61067e30c8a3") : configmap "kube-rbac-proxy" not found
Jul 18 01:49:00.509968 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.509950    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77725796-3a90-4975-aaad-61067e30c8a3-data-volume\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.519980 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.519922    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2sbx\" (UniqueName: \"kubernetes.io/projected/77725796-3a90-4975-aaad-61067e30c8a3-kube-api-access-q2sbx\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:00.526436 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.526308    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bdbxm" podStartSLOduration=2.940248933 podStartE2EDuration="10.526291403s" totalImagesPullingTime="7.58604247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:00.52613016 +0000 UTC m=+45.965627122" watchObservedRunningTime="2026-07-18 01:49:00.526291403 +0000 UTC m=+45.965788334"
Jul 18 01:49:00.749548 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.749504    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:49:00.749717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:00.749583    2552 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:49:01.015597 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.015560    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:01.015790 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:01.015704    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:49:01.015790 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.015713    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:01.015790 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:01.015785    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm podName:77725796-3a90-4975-aaad-61067e30c8a3 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:02.015764826 +0000 UTC m=+47.455261754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9jpv" (UID: "77725796-3a90-4975-aaad-61067e30c8a3") : configmap "kube-rbac-proxy" not found
Jul 18 01:49:01.015936 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:01.015803    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:49:01.015936 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:01.015857    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls podName:77725796-3a90-4975-aaad-61067e30c8a3 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:02.015841097 +0000 UTC m=+47.455338011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9jpv" (UID: "77725796-3a90-4975-aaad-61067e30c8a3") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:49:01.053797 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.053755    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:01.154439 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.154386    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:01.257285 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.257253    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:01.361624 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.361549    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:01.431869 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.431840    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:01.469387 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.469359    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 01:49:01.469864 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.469806    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/0.log"
Jul 18 01:49:01.469864 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.469836    2552 generic.go:356] "Generic (PLEG): container finished" podID="2cbad028-6f01-4a6c-941f-966af50c85c0" containerID="4d98b9320a22f43356866214b3f4f26d3b101d7edfafb76a192d517fb1a8a206" exitCode=255
Jul 18 01:49:01.469971 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.469868    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7" event={"ID":"2cbad028-6f01-4a6c-941f-966af50c85c0","Type":"ContainerDied","Data":"4d98b9320a22f43356866214b3f4f26d3b101d7edfafb76a192d517fb1a8a206"}
Jul 18 01:49:01.469971 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.469929    2552 scope.go:117] "RemoveContainer" containerID="90d3cb2c21339086f9ccd1387a638e9ce13fed20e7b0bc7b148e3f87bdb4a09b"
Jul 18 01:49:01.470203 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.470188    2552 scope.go:117] "RemoveContainer" containerID="4d98b9320a22f43356866214b3f4f26d3b101d7edfafb76a192d517fb1a8a206"
Jul 18 01:49:01.470408 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:01.470384    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-gmrn7_openshift-console-operator(2cbad028-6f01-4a6c-941f-966af50c85c0)\"" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7" podUID="2cbad028-6f01-4a6c-941f-966af50c85c0"
Jul 18 01:49:01.497244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.497202    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:01.779148 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.779109    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:01.847779 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:01.847740    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:02.026538 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:02.026503    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:02.026717 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:02.026643    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:49:02.026717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:02.026664    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:02.026717 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:02.026709    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm podName:77725796-3a90-4975-aaad-61067e30c8a3 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:04.02669096 +0000 UTC m=+49.466187885 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9jpv" (UID: "77725796-3a90-4975-aaad-61067e30c8a3") : configmap "kube-rbac-proxy" not found
Jul 18 01:49:02.026824 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:02.026778    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:49:02.026862 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:02.026827    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls podName:77725796-3a90-4975-aaad-61067e30c8a3 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:04.026813119 +0000 UTC m=+49.466310030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9jpv" (UID: "77725796-3a90-4975-aaad-61067e30c8a3") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:49:02.475283 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:02.475254    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 01:49:02.475668 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:02.475572    2552 scope.go:117] "RemoveContainer" containerID="4d98b9320a22f43356866214b3f4f26d3b101d7edfafb76a192d517fb1a8a206"
Jul 18 01:49:02.475748 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:02.475728    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-gmrn7_openshift-console-operator(2cbad028-6f01-4a6c-941f-966af50c85c0)\"" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7" podUID="2cbad028-6f01-4a6c-941f-966af50c85c0"
Jul 18 01:49:02.540606 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:02.540571    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-strgq_ab7b9b44-0deb-42a5-9bc6-26778ed68ee5/dns-node-resolver/0.log"
Jul 18 01:49:03.494869 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:03.494831    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:03.546607 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:03.546583    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nqcrm_c5ac9a0d-d093-4e02-a9b2-402bddadb797/node-ca/0.log"
Jul 18 01:49:03.595244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:03.595213    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:03.940434 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:03.940405    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bdbxm_1e601610-f330-493b-adf7-13534056efa2/serve-healthcheck-canary/0.log"
Jul 18 01:49:04.044809 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:04.044777    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:04.044994 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:04.044847    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:04.044994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:04.044918    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:49:04.044994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:04.044951    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:49:04.044994 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:04.044985    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm podName:77725796-3a90-4975-aaad-61067e30c8a3 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:08.044968748 +0000 UTC m=+53.484465658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9jpv" (UID: "77725796-3a90-4975-aaad-61067e30c8a3") : configmap "kube-rbac-proxy" not found
Jul 18 01:49:04.045172 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:04.045010    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls podName:77725796-3a90-4975-aaad-61067e30c8a3 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:08.044992009 +0000 UTC m=+53.484488938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9jpv" (UID: "77725796-3a90-4975-aaad-61067e30c8a3") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:49:04.444987 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:04.444948    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:05.053386 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:05.053350    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:06.266578 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:06.266515    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:49:06.266578 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:06.266584    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:06.266619    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:06.266658    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:06.266702    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:06.266733    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:06.266753    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266697    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266812    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266791    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:22.266769265 +0000 UTC m=+67.706266174 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : configmap references non-existent config key: service-ca.crt
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266837    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266849    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls podName:874287f6-35db-4ecb-9213-64ef5ed15dfa nodeName:}" failed. No retries permitted until 2026-07-18 01:49:22.266836916 +0000 UTC m=+67.706333829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls") pod "dns-default-dvgh9" (UID: "874287f6-35db-4ecb-9213-64ef5ed15dfa") : secret "dns-default-metrics-tls" not found
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266785    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266860    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266891    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs podName:93ed0884-1025-4a1c-b573-dfa09a741a37 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:22.266875456 +0000 UTC m=+67.706372368 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs") pod "router-default-6c89c86f68-22ccf" (UID: "93ed0884-1025-4a1c-b573-dfa09a741a37") : secret "router-metrics-certs-default" not found
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266906    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert podName:7081f4df-c702-41ab-b078-d62252a5b207 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:22.266899632 +0000 UTC m=+67.706396542 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hzgdx" (UID: "7081f4df-c702-41ab-b078-d62252a5b207") : secret "networking-console-plugin-cert" not found
Jul 18 01:49:06.267087 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266919    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:49:06.267641 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266928    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls podName:46a417f1-eed7-4643-a01a-0cb5029dcc78 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:22.26691216 +0000 UTC m=+67.706409070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-m5m8z" (UID: "46a417f1-eed7-4643-a01a-0cb5029dcc78") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:49:06.267641 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266864    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-99ff6f44d-sngjf: secret "image-registry-tls" not found
Jul 18 01:49:06.267641 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266968    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls podName:ae958a97-679e-4c2b-bb4d-6c8719563be1 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:22.26695904 +0000 UTC m=+67.706455952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls") pod "image-registry-99ff6f44d-sngjf" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1") : secret "image-registry-tls" not found
Jul 18 01:49:06.267641 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:06.266982    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls podName:b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab nodeName:}" failed. No retries permitted until 2026-07-18 01:49:22.266975457 +0000 UTC m=+67.706472370 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-d6m27" (UID: "b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab") : secret "samples-operator-tls" not found
Jul 18 01:49:07.001445 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:07.001404    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:07.069292 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:07.069266    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:07.622112 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:07.622080    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:07.721848 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:07.721817    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:07.822711 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:07.822680    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:07.923350 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:07.923315    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:08.081320 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:08.081281    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:08.081508 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:08.081390    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:08.081508 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:08.081503    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:49:08.081591 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:08.081560    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls podName:77725796-3a90-4975-aaad-61067e30c8a3 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:16.081545734 +0000 UTC m=+61.521042644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9jpv" (UID: "77725796-3a90-4975-aaad-61067e30c8a3") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:49:08.081906 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:08.081889    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77725796-3a90-4975-aaad-61067e30c8a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:08.142477 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:08.142442    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:08.287788 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:08.287681    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:08.366909 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:08.366883    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:09.305905 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:09.305870    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:09.712497 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:09.712467    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:09.906413 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:09.906383    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:10.107406 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:10.107335    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:10.505120 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:10.505084    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:10.749346 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:10.749296    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:49:10.749578 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:10.749376    2552 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:49:10.749845 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:10.749828    2552 scope.go:117] "RemoveContainer" containerID="4d98b9320a22f43356866214b3f4f26d3b101d7edfafb76a192d517fb1a8a206"
Jul 18 01:49:10.909832 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:10.909777    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:11.311885 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:11.311854    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:11.412964 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:11.412923    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:11.504496 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:11.504467    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 01:49:11.504676 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:11.504558    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7" event={"ID":"2cbad028-6f01-4a6c-941f-966af50c85c0","Type":"ContainerStarted","Data":"ab3f27c671118b81d713a8183cbe7d4ad4179fb25c960a0d67c57b60fa2bf51c"}
Jul 18 01:49:11.504906 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:11.504888    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:49:11.509663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:11.509634    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7"
Jul 18 01:49:11.511914 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:11.511890    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:11.522278 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:11.522229    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-gmrn7" podStartSLOduration=26.003235109 podStartE2EDuration="33.522212762s" totalImagesPullingTime="7.518977653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:11.521743008 +0000 UTC m=+56.961239940" watchObservedRunningTime="2026-07-18 01:49:11.522212762 +0000 UTC m=+56.961709694"
Jul 18 01:49:11.571355 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:11.571251    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:11.908404 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:11.908374    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:12.010574 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:12.010543    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:12.110410 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:12.110378    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:12.211689 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:12.211618    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:12.351128 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:12.351099    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:12.434649 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:12.434609    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:12.706900 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:12.706871    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:13.170495 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:13.170461    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:13.300838 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:13.300806    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:13.401545 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:13.401507    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:13.505763 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:13.505679    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:13.608094 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:13.608050    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:13.710500 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:13.710472    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:14.104852 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:14.104821    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:14.502870 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:14.502836    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:14.721029 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:14.720998    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:15.311994 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:15.311943    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:15.507582 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:15.507532    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:15.626655 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:15.626546    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:15.708867 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:15.708821    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:15.913511 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:15.913477    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:16.156806 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:16.156769    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:16.159226 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:16.159201    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77725796-3a90-4975-aaad-61067e30c8a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9jpv\" (UID: \"77725796-3a90-4975-aaad-61067e30c8a3\") " pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:16.176981 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:16.176904    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jz89w\""
Jul 18 01:49:16.184859 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:16.184838    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k9jpv"
Jul 18 01:49:16.315359 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:16.315324    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k9jpv"]
Jul 18 01:49:16.320062 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:49:16.320034    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77725796_3a90_4975_aaad_61067e30c8a3.slice/crio-851b0a055757685218c133a67f6cc4aa8ec3f54890256b92286aa4cfecc1f58e WatchSource:0}: Error finding container 851b0a055757685218c133a67f6cc4aa8ec3f54890256b92286aa4cfecc1f58e: Status 404 returned error can't find the container with id 851b0a055757685218c133a67f6cc4aa8ec3f54890256b92286aa4cfecc1f58e
Jul 18 01:49:16.364290 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:16.364260    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nt7rh"
Jul 18 01:49:16.520153 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:16.520062    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9jpv" event={"ID":"77725796-3a90-4975-aaad-61067e30c8a3","Type":"ContainerStarted","Data":"851b0a055757685218c133a67f6cc4aa8ec3f54890256b92286aa4cfecc1f58e"}
Jul 18 01:49:17.106921 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:17.106860    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:17.308167 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:17.308132    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:17.503772 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:17.503723    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:17.717857 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:17.717823    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:17.921593 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:17.921555    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:18.504648 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:18.504613    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:18.705476 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:18.705440    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:20.701593 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.701551    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:49:20.702036 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.701652    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxq6c\" (UniqueName: \"kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c\") pod \"network-check-target-42wx4\" (UID: \"bb0b5223-b563-4075-8406-c096402a16c5\") " pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:49:20.704805 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.704779    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 01:49:20.704953 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.704864    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxq6c\" (UniqueName: \"kubernetes.io/projected/bb0b5223-b563-4075-8406-c096402a16c5-kube-api-access-cxq6c\") pod \"network-check-target-42wx4\" (UID: \"bb0b5223-b563-4075-8406-c096402a16c5\") " pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:49:20.714966 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.714909    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de548f6-694f-425d-9ee4-5ac3df26aed4-metrics-certs\") pod \"network-metrics-daemon-fgqqf\" (UID: \"4de548f6-694f-425d-9ee4-5ac3df26aed4\") " pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:49:20.802998 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.802956    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:49:20.806265 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.806242    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 01:49:20.815999 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.815972    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45860c98-3146-470d-8c2c-3feb65e32e10-original-pull-secret\") pod \"global-pull-secret-syncer-zr5mn\" (UID: \"45860c98-3146-470d-8c2c-3feb65e32e10\") " pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:49:20.886411 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.886374    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zr5mn"
Jul 18 01:49:20.898984 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.898960    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mcksx\""
Jul 18 01:49:20.905456 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.905406    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2rqj2\""
Jul 18 01:49:20.906973 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.906949    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgqqf"
Jul 18 01:49:20.914015 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:20.913995    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:49:21.254766 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.254737    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zr5mn"]
Jul 18 01:49:21.257896 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:49:21.257865    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45860c98_3146_470d_8c2c_3feb65e32e10.slice/crio-d7e6ade75bfa89abf7497672b94dca59d17559199a104b39be94cd51ba6eb797 WatchSource:0}: Error finding container d7e6ade75bfa89abf7497672b94dca59d17559199a104b39be94cd51ba6eb797: Status 404 returned error can't find the container with id d7e6ade75bfa89abf7497672b94dca59d17559199a104b39be94cd51ba6eb797
Jul 18 01:49:21.274611 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.274579    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-42wx4"]
Jul 18 01:49:21.278374 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:49:21.278338    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0b5223_b563_4075_8406_c096402a16c5.slice/crio-dce5696f8c4241a60707fa9eb183f11dab15861c4a99fd0d8ef9fdba116907c0 WatchSource:0}: Error finding container dce5696f8c4241a60707fa9eb183f11dab15861c4a99fd0d8ef9fdba116907c0: Status 404 returned error can't find the container with id dce5696f8c4241a60707fa9eb183f11dab15861c4a99fd0d8ef9fdba116907c0
Jul 18 01:49:21.381889 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.381810    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:21.475510 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.475353    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fgqqf"]
Jul 18 01:49:21.477990 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:49:21.477959    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de548f6_694f_425d_9ee4_5ac3df26aed4.slice/crio-1c9ac2b8694a759b00904bdf5addb8e8c04aa94f091b4bfe59fb58e67e426364 WatchSource:0}: Error finding container 1c9ac2b8694a759b00904bdf5addb8e8c04aa94f091b4bfe59fb58e67e426364: Status 404 returned error can't find the container with id 1c9ac2b8694a759b00904bdf5addb8e8c04aa94f091b4bfe59fb58e67e426364
Jul 18 01:49:21.481335 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.481305    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:21.541102 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.541066    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zr5mn" event={"ID":"45860c98-3146-470d-8c2c-3feb65e32e10","Type":"ContainerStarted","Data":"d7e6ade75bfa89abf7497672b94dca59d17559199a104b39be94cd51ba6eb797"}
Jul 18 01:49:21.542236 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.542210    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fgqqf" event={"ID":"4de548f6-694f-425d-9ee4-5ac3df26aed4","Type":"ContainerStarted","Data":"1c9ac2b8694a759b00904bdf5addb8e8c04aa94f091b4bfe59fb58e67e426364"}
Jul 18 01:49:21.543868 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.543843    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-42wx4" event={"ID":"bb0b5223-b563-4075-8406-c096402a16c5","Type":"ContainerStarted","Data":"3789e1b6cf6e3ed0ffed6054f9bfef023a09be7ea79aaa7d2bb025a8f0958208"}
Jul 18 01:49:21.543993 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.543900    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-42wx4" event={"ID":"bb0b5223-b563-4075-8406-c096402a16c5","Type":"ContainerStarted","Data":"dce5696f8c4241a60707fa9eb183f11dab15861c4a99fd0d8ef9fdba116907c0"}
Jul 18 01:49:21.543993 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.543943    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:49:21.545441 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.545393    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9jpv" event={"ID":"77725796-3a90-4975-aaad-61067e30c8a3","Type":"ContainerStarted","Data":"ae1bc18ef724de544e02c5f6c20000d4ab806827e3f31c1ff29eb7ea399412ad"}
Jul 18 01:49:21.560745 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:21.560688    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-42wx4" podStartSLOduration=66.560671889 podStartE2EDuration="1m6.560671889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:21.559821974 +0000 UTC m=+66.999318907" watchObservedRunningTime="2026-07-18 01:49:21.560671889 +0000 UTC m=+67.000168820"
Jul 18 01:49:22.324625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.324190    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:49:22.324625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.324245    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:49:22.324625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.324292    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:49:22.324625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.324333    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:49:22.324625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.324393    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:49:22.324625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.324454    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:49:22.324625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.324485    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:49:22.325648 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.325031    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ed0884-1025-4a1c-b573-dfa09a741a37-service-ca-bundle\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:49:22.329134 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.329096    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7081f4df-c702-41ab-b078-d62252a5b207-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hzgdx\" (UID: \"7081f4df-c702-41ab-b078-d62252a5b207\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:49:22.330839 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.330807    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93ed0884-1025-4a1c-b573-dfa09a741a37-metrics-certs\") pod \"router-default-6c89c86f68-22ccf\" (UID: \"93ed0884-1025-4a1c-b573-dfa09a741a37\") " pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:49:22.331269 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.331230    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls\") pod \"image-registry-99ff6f44d-sngjf\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") " pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:49:22.331526 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.331501    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-d6m27\" (UID: \"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:49:22.333183 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.333159    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/874287f6-35db-4ecb-9213-64ef5ed15dfa-metrics-tls\") pod \"dns-default-dvgh9\" (UID: \"874287f6-35db-4ecb-9213-64ef5ed15dfa\") " pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:49:22.334287 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.334247    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/46a417f1-eed7-4643-a01a-0cb5029dcc78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-m5m8z\" (UID: \"46a417f1-eed7-4643-a01a-0cb5029dcc78\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:49:22.510822 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.510786    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tsvv8\""
Jul 18 01:49:22.518988 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.518774    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:49:22.519943 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.519919    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ml9cq\""
Jul 18 01:49:22.528099 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.528070    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"
Jul 18 01:49:22.542091 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.541882    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-95t94\""
Jul 18 01:49:22.549798 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.549776    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"
Jul 18 01:49:22.558305 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.558108    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s6gqx\""
Jul 18 01:49:22.568840 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.568603    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:49:22.573767 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.573743    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mfpwx\""
Jul 18 01:49:22.581473 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.581392    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:49:22.602951 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.602907    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xnmph\""
Jul 18 01:49:22.611866 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:22.611666    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"
Jul 18 01:49:23.134807 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:23.134773    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-99ff6f44d-sngjf"]
Jul 18 01:49:23.196189 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:23.196115    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dvgh9"]
Jul 18 01:49:23.352010 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:23.351987    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27"]
Jul 18 01:49:23.354561 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:23.354534    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c89c86f68-22ccf"]
Jul 18 01:49:23.355786 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:23.355481    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx"]
Jul 18 01:49:23.359995 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:23.359972    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z"]
Jul 18 01:49:23.948597 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:23.948556    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:24.037040 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:24.036990    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:24.106157 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:24.106110    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:24.506374 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:49:24.506327    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod874287f6_35db_4ecb_9213_64ef5ed15dfa.slice/crio-afbf8216458e11735ac299b84a59524da8a8e8097786ab607a37d327f4ba6437 WatchSource:0}: Error finding container afbf8216458e11735ac299b84a59524da8a8e8097786ab607a37d327f4ba6437: Status 404 returned error can't find the container with id afbf8216458e11735ac299b84a59524da8a8e8097786ab607a37d327f4ba6437
Jul 18 01:49:24.507981 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:49:24.507950    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46a417f1_eed7_4643_a01a_0cb5029dcc78.slice/crio-24aeb5fcae8e93ae9ef9ce6794009927e80fab1e6f9a4f9b7f4fe8ee8738f755 WatchSource:0}: Error finding container 24aeb5fcae8e93ae9ef9ce6794009927e80fab1e6f9a4f9b7f4fe8ee8738f755: Status 404 returned error can't find the container with id 24aeb5fcae8e93ae9ef9ce6794009927e80fab1e6f9a4f9b7f4fe8ee8738f755
Jul 18 01:49:24.557549 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:24.557460    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dvgh9" event={"ID":"874287f6-35db-4ecb-9213-64ef5ed15dfa","Type":"ContainerStarted","Data":"afbf8216458e11735ac299b84a59524da8a8e8097786ab607a37d327f4ba6437"}
Jul 18 01:49:24.559315 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:24.559285    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z" event={"ID":"46a417f1-eed7-4643-a01a-0cb5029dcc78","Type":"ContainerStarted","Data":"24aeb5fcae8e93ae9ef9ce6794009927e80fab1e6f9a4f9b7f4fe8ee8738f755"}
Jul 18 01:49:25.564084 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:25.564044    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27" event={"ID":"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab","Type":"ContainerStarted","Data":"3547cc6806cded67f2a616f13892a3d44b521bd020d07996a40c0872b9e9f96a"}
Jul 18 01:49:25.945767 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:49:25.945721    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7081f4df_c702_41ab_b078_d62252a5b207.slice/crio-3c50ff3d937f009fc53891a7d0071cc16e8d3ed6f6a7f9efe7e7085d4799a8a4 WatchSource:0}: Error finding container 3c50ff3d937f009fc53891a7d0071cc16e8d3ed6f6a7f9efe7e7085d4799a8a4: Status 404 returned error can't find the container with id 3c50ff3d937f009fc53891a7d0071cc16e8d3ed6f6a7f9efe7e7085d4799a8a4
Jul 18 01:49:25.947144 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:49:25.947110    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ed0884_1025_4a1c_b573_dfa09a741a37.slice/crio-0de179edcda881a1721f38c3ccbe0bb21d8e8552423f756b0fbd2275cdd439c1 WatchSource:0}: Error finding container 0de179edcda881a1721f38c3ccbe0bb21d8e8552423f756b0fbd2275cdd439c1: Status 404 returned error can't find the container with id 0de179edcda881a1721f38c3ccbe0bb21d8e8552423f756b0fbd2275cdd439c1
Jul 18 01:49:25.947451 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:49:25.947363    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae958a97_679e_4c2b_bb4d_6c8719563be1.slice/crio-ec94e213ba5d7fb1bb663a2d10b2716277bbb2c6ce864c828c5ca9dc0e82cefc WatchSource:0}: Error finding container ec94e213ba5d7fb1bb663a2d10b2716277bbb2c6ce864c828c5ca9dc0e82cefc: Status 404 returned error can't find the container with id ec94e213ba5d7fb1bb663a2d10b2716277bbb2c6ce864c828c5ca9dc0e82cefc
Jul 18 01:49:25.996526 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:25.996488    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:26.090534 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:26.090505    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:26.463509 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:26.463476    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:26.564322 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:26.564285    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:26.568845 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:26.568809    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf" event={"ID":"ae958a97-679e-4c2b-bb4d-6c8719563be1","Type":"ContainerStarted","Data":"ec94e213ba5d7fb1bb663a2d10b2716277bbb2c6ce864c828c5ca9dc0e82cefc"}
Jul 18 01:49:26.570118 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:26.570092    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c89c86f68-22ccf" event={"ID":"93ed0884-1025-4a1c-b573-dfa09a741a37","Type":"ContainerStarted","Data":"0de179edcda881a1721f38c3ccbe0bb21d8e8552423f756b0fbd2275cdd439c1"}
Jul 18 01:49:26.571356 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:26.571333    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx" event={"ID":"7081f4df-c702-41ab-b078-d62252a5b207","Type":"ContainerStarted","Data":"3c50ff3d937f009fc53891a7d0071cc16e8d3ed6f6a7f9efe7e7085d4799a8a4"}
Jul 18 01:49:27.582811 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.582770    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9jpv" event={"ID":"77725796-3a90-4975-aaad-61067e30c8a3","Type":"ContainerStarted","Data":"ee30a8c6be96f744749aa8c9fbef0dc09aa26431a8371dafce822c49e0adc8ca"}
Jul 18 01:49:27.586562 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.586525    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c89c86f68-22ccf" event={"ID":"93ed0884-1025-4a1c-b573-dfa09a741a37","Type":"ContainerStarted","Data":"13b3983aabfdf3260428ed67f041aeaf72cea72b46a1bc2e8c4a8a02af541449"}
Jul 18 01:49:27.590595 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.590541    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zr5mn" event={"ID":"45860c98-3146-470d-8c2c-3feb65e32e10","Type":"ContainerStarted","Data":"6da8a4aeb089290edc6359e7966407d4b9aa08c40dd365773f69336932441dc3"}
Jul 18 01:49:27.596889 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.596254    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fgqqf" event={"ID":"4de548f6-694f-425d-9ee4-5ac3df26aed4","Type":"ContainerStarted","Data":"363c2693bb7a3c9150478bcd41dd76b4ce3ed9833e6cd3fcbf10d7eb6740e068"}
Jul 18 01:49:27.596889 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.596312    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fgqqf" event={"ID":"4de548f6-694f-425d-9ee4-5ac3df26aed4","Type":"ContainerStarted","Data":"33a077d6e935e83dc99b88d7672cbaaf4fc10de18c6d0f1be3bbcbd8c6b4e721"}
Jul 18 01:49:27.599975 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.599440    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf" event={"ID":"ae958a97-679e-4c2b-bb4d-6c8719563be1","Type":"ContainerStarted","Data":"6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af"}
Jul 18 01:49:27.599975 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.599935    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:49:27.623464 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.623388    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fgqqf" podStartSLOduration=67.464347687 podStartE2EDuration="1m12.623367313s" totalImagesPullingTime="5.159019626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:27.622981885 +0000 UTC m=+73.062478819" watchObservedRunningTime="2026-07-18 01:49:27.623367313 +0000 UTC m=+73.062864246"
Jul 18 01:49:27.624317 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.624282    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6c89c86f68-22ccf" podStartSLOduration=49.624270471 podStartE2EDuration="49.624270471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:27.606624897 +0000 UTC m=+73.046121831" watchObservedRunningTime="2026-07-18 01:49:27.624270471 +0000 UTC m=+73.063767423"
Jul 18 01:49:27.637818 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.637757    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zr5mn" podStartSLOduration=66.198404447 podStartE2EDuration="1m11.637738356s" totalImagesPullingTime="5.439333909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:27.637123849 +0000 UTC m=+73.076620794" watchObservedRunningTime="2026-07-18 01:49:27.637738356 +0000 UTC m=+73.077235289"
Jul 18 01:49:27.656558 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.656497    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf" podStartSLOduration=71.656475221 podStartE2EDuration="1m11.656475221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:27.654456209 +0000 UTC m=+73.093953167" watchObservedRunningTime="2026-07-18 01:49:27.656475221 +0000 UTC m=+73.095972155"
Jul 18 01:49:27.724957 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.724923    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:27.826563 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.826524    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:27.992474 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:27.992446    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:28.068219 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:28.068175    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:28.227116 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:28.227067    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:28.285450 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:28.285217    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:28.582637 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:28.582558    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:49:28.585324 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:28.585302    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:49:28.605318 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:28.605290    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:49:28.606833 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:28.606810    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6c89c86f68-22ccf"
Jul 18 01:49:29.634575 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:29.634546    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:29.697726 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:29.697691    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:29.783512 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:29.783478    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:31.392538 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:31.392495    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:31.491685 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:31.491649    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:32.400227 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.400194    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:32.504656 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.504568    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:32.620265 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.620224    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27" event={"ID":"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab","Type":"ContainerStarted","Data":"ef18ac1afddc8664332dcc928f8abbff480251377186c4d787a7b29b08ad2a77"}
Jul 18 01:49:32.620496 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.620290    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27" event={"ID":"b3fbe760-c8f6-4bc7-b7eb-cacae63a8cab","Type":"ContainerStarted","Data":"a94052ccbcc9f58d33c2b8751f8d8234291d789eacd02c794a430ed1dfa990b8"}
Jul 18 01:49:32.622503 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.622470    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9jpv" event={"ID":"77725796-3a90-4975-aaad-61067e30c8a3","Type":"ContainerStarted","Data":"48cb6a00b41b0e31b2c8ea0237086f3ce9929ad49e61ccd77c5e82247e975937"}
Jul 18 01:49:32.624076 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.624053    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx" event={"ID":"7081f4df-c702-41ab-b078-d62252a5b207","Type":"ContainerStarted","Data":"394c6167c6b50b2f07dfc99f8afcdca5ab10e3945783b64e0847d2b4e5e1a7c2"}
Jul 18 01:49:32.625635 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.625616    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z" event={"ID":"46a417f1-eed7-4643-a01a-0cb5029dcc78","Type":"ContainerStarted","Data":"1bbcf8cbffd62fb5e1393a865017f4df66b7ff1886f9628d6b7f1ba370b2a8f5"}
Jul 18 01:49:32.627302 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.627283    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dvgh9" event={"ID":"874287f6-35db-4ecb-9213-64ef5ed15dfa","Type":"ContainerStarted","Data":"4b2b0f3e666631e0f5dac0cdceb6f06cc5661dfec0e9b968db0f7b819be5ea6f"}
Jul 18 01:49:32.627386 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.627317    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dvgh9" event={"ID":"874287f6-35db-4ecb-9213-64ef5ed15dfa","Type":"ContainerStarted","Data":"ffc7f3d961f4ce75eb13cb77e96fd46a5da054e54b3189906f1840efcdd8a531"}
Jul 18 01:49:32.627386 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.627352    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:49:32.639071 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.639023    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-d6m27" podStartSLOduration=47.30023992 podStartE2EDuration="54.639006275s" totalImagesPullingTime="7.338766355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:32.637836637 +0000 UTC m=+78.077333570" watchObservedRunningTime="2026-07-18 01:49:32.639006275 +0000 UTC m=+78.078503207"
Jul 18 01:49:32.657911 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.657842    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k9jpv" podStartSLOduration=16.993227163 podStartE2EDuration="32.657825065s" totalImagesPullingTime="15.664597902s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:49:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:32.657219702 +0000 UTC m=+78.096716631" watchObservedRunningTime="2026-07-18 01:49:32.657825065 +0000 UTC m=+78.097321997"
Jul 18 01:49:32.674273 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.674222    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dvgh9" podStartSLOduration=35.273616859 podStartE2EDuration="42.674204803s" totalImagesPullingTime="7.400587944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:32.672861123 +0000 UTC m=+78.112358053" watchObservedRunningTime="2026-07-18 01:49:32.674204803 +0000 UTC m=+78.113701725"
Jul 18 01:49:32.689054 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.688990    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-m5m8z" podStartSLOduration=47.296324397 podStartE2EDuration="54.68896811s" totalImagesPullingTime="7.392643713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:32.68714403 +0000 UTC m=+78.126640979" watchObservedRunningTime="2026-07-18 01:49:32.68896811 +0000 UTC m=+78.128465042"
Jul 18 01:49:32.706485 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.706406    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hzgdx" podStartSLOduration=38.7541806 podStartE2EDuration="44.706385741s" totalImagesPullingTime="5.952205141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:32.700344276 +0000 UTC m=+78.139841208" watchObservedRunningTime="2026-07-18 01:49:32.706385741 +0000 UTC m=+78.145882674"
Jul 18 01:49:32.798832 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.798741    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:32.867811 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.867777    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:32.930918 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:32.930891    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:33.114771 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:33.114682    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:33.275525 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:33.275495    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:33.376898 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:33.376811    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:33.438494 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:33.438460    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:34.996573 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:34.996540    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:35.063802 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:35.063771    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:35.401125 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:35.401097    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:35.466441 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:35.466396    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:37.254495 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:37.254459    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:37.349583 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:37.349549    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:37.541816 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:37.541737    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:37.677723 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:37.677688    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:37.740766 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:37.740737    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:39.585509 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:39.585450    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:39.687515 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:39.687480    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:41.447012 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:41.446966    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:41.549101 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:41.549071    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:41.791013 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:41.790940    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:42.634893 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:42.634860    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dvgh9"
Jul 18 01:49:42.785542 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:42.785510    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:43.240623 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:43.240592    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:43.341605 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:43.341572    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:43.618245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:43.618153    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:43.717447 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:43.717391    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:44.512299 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:44.512264    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:44.612547 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:44.612514    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:46.189046 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:46.189018    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-99ff6f44d-sngjf"]
Jul 18 01:49:46.193413 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:46.193386    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:49:46.289752 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:46.289721    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:46.357066 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:46.357034    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:46.419674 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:46.419637    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:47.072569 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:47.072537    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:49.657476 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.657418    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:49.872656 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.872625    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:49.915788 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.915706    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tzlxq"]
Jul 18 01:49:49.918796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.918775    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:49.921071 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.921036    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 18 01:49:49.921222 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.921124    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 18 01:49:49.921222 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.921175    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 18 01:49:49.921783 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.921765    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fc5s8\""
Jul 18 01:49:49.921783 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.921771    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 18 01:49:49.949669 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.949641    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-tls\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:49.949669 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.949674    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:49.949898 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.949695    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-textfile\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:49.949898 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.949765    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-accelerators-collector-config\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:49.949898 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.949792    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-wtmp\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:49.949898 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.949843    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cecc90db-591c-451c-b527-bb1628faf2fc-root\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:49.949898 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.949869    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cecc90db-591c-451c-b527-bb1628faf2fc-sys\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:49.949898 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.949890    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72wcz\" (UniqueName: \"kubernetes.io/projected/cecc90db-591c-451c-b527-bb1628faf2fc-kube-api-access-72wcz\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:49.950092 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.949912    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cecc90db-591c-451c-b527-bb1628faf2fc-metrics-client-ca\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:49.963768 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:49.963720    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:50.050763 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.050722    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-accelerators-collector-config\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.050763 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.050766    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-wtmp\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.050799    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cecc90db-591c-451c-b527-bb1628faf2fc-root\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.050828    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cecc90db-591c-451c-b527-bb1628faf2fc-sys\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.050850    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72wcz\" (UniqueName: \"kubernetes.io/projected/cecc90db-591c-451c-b527-bb1628faf2fc-kube-api-access-72wcz\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.050877    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cecc90db-591c-451c-b527-bb1628faf2fc-metrics-client-ca\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.050923    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cecc90db-591c-451c-b527-bb1628faf2fc-root\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.050955    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-tls\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.050993    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.051002    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-wtmp\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.051022    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-textfile\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.050933    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cecc90db-591c-451c-b527-bb1628faf2fc-sys\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051560 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:50.051039    2552 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 18 01:49:50.051560 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:50.051099    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-tls podName:cecc90db-591c-451c-b527-bb1628faf2fc nodeName:}" failed. No retries permitted until 2026-07-18 01:49:50.551077339 +0000 UTC m=+95.990574254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-tls") pod "node-exporter-tzlxq" (UID: "cecc90db-591c-451c-b527-bb1628faf2fc") : secret "node-exporter-tls" not found
Jul 18 01:49:50.051560 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.051276    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-textfile\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051560 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.051361    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-accelerators-collector-config\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.051560 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.051498    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cecc90db-591c-451c-b527-bb1628faf2fc-metrics-client-ca\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.053352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.053330    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.061730 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.061705    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72wcz\" (UniqueName: \"kubernetes.io/projected/cecc90db-591c-451c-b527-bb1628faf2fc-kube-api-access-72wcz\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.157082 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.157049    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:50.258031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.257950    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:50.385783 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.385749    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:50.484850 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.484808    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:50.554102 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.554011    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-tls\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:50.554260 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:50.554160    2552 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 18 01:49:50.554260 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:49:50.554222    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-tls podName:cecc90db-591c-451c-b527-bb1628faf2fc nodeName:}" failed. No retries permitted until 2026-07-18 01:49:51.554207104 +0000 UTC m=+96.993704014 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-tls") pod "node-exporter-tzlxq" (UID: "cecc90db-591c-451c-b527-bb1628faf2fc") : secret "node-exporter-tls" not found
Jul 18 01:49:50.667655 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.667624    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:50.768476 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.768444    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:50.871781 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.871698    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:50.972651 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:50.972620    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:51.049151 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:51.049122    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:51.132304 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:51.132200    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:51.193362 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:51.193306    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:51.563505 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:51.563474    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-tls\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:51.565901 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:51.565877    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cecc90db-591c-451c-b527-bb1628faf2fc-node-exporter-tls\") pod \"node-exporter-tzlxq\" (UID: \"cecc90db-591c-451c-b527-bb1628faf2fc\") " pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:51.728681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:51.728642    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tzlxq"
Jul 18 01:49:51.775952 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:49:51.775919    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcecc90db_591c_451c_b527_bb1628faf2fc.slice/crio-4a9d13f40f126b011d669d7e6000bf12e471aa2f700e15012ac86ce050927f46 WatchSource:0}: Error finding container 4a9d13f40f126b011d669d7e6000bf12e471aa2f700e15012ac86ce050927f46: Status 404 returned error can't find the container with id 4a9d13f40f126b011d669d7e6000bf12e471aa2f700e15012ac86ce050927f46
Jul 18 01:49:52.552577 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:52.552550    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-42wx4"
Jul 18 01:49:52.703500 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:52.703457    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tzlxq" event={"ID":"cecc90db-591c-451c-b527-bb1628faf2fc","Type":"ContainerStarted","Data":"4a9d13f40f126b011d669d7e6000bf12e471aa2f700e15012ac86ce050927f46"}
Jul 18 01:49:52.857397 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:52.857316    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:52.959374 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:52.959335    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:53.057886 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:53.057854    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:53.469049 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:53.469018    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:53.709041 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:53.709003    2552 generic.go:356] "Generic (PLEG): container finished" podID="cecc90db-591c-451c-b527-bb1628faf2fc" containerID="33b447b87ca848d4eb623dafbe2c2e300cf8014b8aa717072c84ca125e18f76c" exitCode=0
Jul 18 01:49:53.709221 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:53.709085    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tzlxq" event={"ID":"cecc90db-591c-451c-b527-bb1628faf2fc","Type":"ContainerDied","Data":"33b447b87ca848d4eb623dafbe2c2e300cf8014b8aa717072c84ca125e18f76c"}
Jul 18 01:49:54.689311 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:54.689265    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:54.715326 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:54.715287    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tzlxq" event={"ID":"cecc90db-591c-451c-b527-bb1628faf2fc","Type":"ContainerStarted","Data":"5fd423242f5f85690df9562f048ffbb48717c3cb67c624838d8d8a59403556bc"}
Jul 18 01:49:54.715513 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:54.715392    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tzlxq" event={"ID":"cecc90db-591c-451c-b527-bb1628faf2fc","Type":"ContainerStarted","Data":"9b86fbdebea888545d4c4a74e3adb367591ab9a7836ad0d5011f1f0689651fdc"}
Jul 18 01:49:54.742245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:54.742190    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tzlxq" podStartSLOduration=4.276238197 podStartE2EDuration="5.742171292s" totalImagesPullingTime="1.345139677s" totalInitContainerRuntime="120.793418ms" isStatefulPod=true podCreationTimestamp="2026-07-18 01:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:54.740454458 +0000 UTC m=+100.179951418" watchObservedRunningTime="2026-07-18 01:49:54.742171292 +0000 UTC m=+100.181668224"
Jul 18 01:49:54.774289 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:54.774258    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:54.874182 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:54.874145    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:55.460852 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:55.460816    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:56.151964 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:56.151916    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:56.450266 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:56.450176    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:56.532911 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:56.532881    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:58.132829 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:58.132791    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:58.233978 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:58.233937    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:59.672136 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:59.672102    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:49:59.725780 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:49:59.725745    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:05.933282 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:05.933247    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:06.027715 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:06.027676    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:11.209575 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.209509    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf" podUID="ae958a97-679e-4c2b-bb4d-6c8719563be1" containerName="registry" containerID="cri-o://6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af" gracePeriod=30
Jul 18 01:50:11.474160 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.474134    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:50:11.510302 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.510264    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-trusted-ca\") pod \"ae958a97-679e-4c2b-bb4d-6c8719563be1\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") "
Jul 18 01:50:11.510302 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.510307    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls\") pod \"ae958a97-679e-4c2b-bb4d-6c8719563be1\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") "
Jul 18 01:50:11.510584 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.510331    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ae958a97-679e-4c2b-bb4d-6c8719563be1-ca-trust-extracted\") pod \"ae958a97-679e-4c2b-bb4d-6c8719563be1\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") "
Jul 18 01:50:11.510584 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.510500    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-certificates\") pod \"ae958a97-679e-4c2b-bb4d-6c8719563be1\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") "
Jul 18 01:50:11.510584 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.510542    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-bound-sa-token\") pod \"ae958a97-679e-4c2b-bb4d-6c8719563be1\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") "
Jul 18 01:50:11.510584 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.510580    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-image-registry-private-configuration\") pod \"ae958a97-679e-4c2b-bb4d-6c8719563be1\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") "
Jul 18 01:50:11.510809 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.510613    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7znq2\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-kube-api-access-7znq2\") pod \"ae958a97-679e-4c2b-bb4d-6c8719563be1\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") "
Jul 18 01:50:11.510809 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.510662    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-installation-pull-secrets\") pod \"ae958a97-679e-4c2b-bb4d-6c8719563be1\" (UID: \"ae958a97-679e-4c2b-bb4d-6c8719563be1\") "
Jul 18 01:50:11.510919 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.510862    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ae958a97-679e-4c2b-bb4d-6c8719563be1" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:50:11.511108 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.511078    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ae958a97-679e-4c2b-bb4d-6c8719563be1" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:50:11.513712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.513671    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-kube-api-access-7znq2" (OuterVolumeSpecName: "kube-api-access-7znq2") pod "ae958a97-679e-4c2b-bb4d-6c8719563be1" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1"). InnerVolumeSpecName "kube-api-access-7znq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:50:11.513712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.513680    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ae958a97-679e-4c2b-bb4d-6c8719563be1" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:11.513882 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.513764    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ae958a97-679e-4c2b-bb4d-6c8719563be1" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:50:11.513882 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.513842    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ae958a97-679e-4c2b-bb4d-6c8719563be1" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:50:11.513882 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.513867    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ae958a97-679e-4c2b-bb4d-6c8719563be1" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:50:11.519588 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.519558    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae958a97-679e-4c2b-bb4d-6c8719563be1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ae958a97-679e-4c2b-bb4d-6c8719563be1" (UID: "ae958a97-679e-4c2b-bb4d-6c8719563be1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:50:11.611478 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.611408    2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-trusted-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:50:11.611478 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.611477    2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-tls\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:50:11.611668 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.611493    2552 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ae958a97-679e-4c2b-bb4d-6c8719563be1-ca-trust-extracted\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:50:11.611668 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.611508    2552 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ae958a97-679e-4c2b-bb4d-6c8719563be1-registry-certificates\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:50:11.611668 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.611524    2552 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-bound-sa-token\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:50:11.611668 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.611540    2552 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-image-registry-private-configuration\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:50:11.611668 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.611555    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7znq2\" (UniqueName: \"kubernetes.io/projected/ae958a97-679e-4c2b-bb4d-6c8719563be1-kube-api-access-7znq2\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:50:11.611668 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.611570    2552 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ae958a97-679e-4c2b-bb4d-6c8719563be1-installation-pull-secrets\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:50:11.779340 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.779249    2552 generic.go:356] "Generic (PLEG): container finished" podID="ae958a97-679e-4c2b-bb4d-6c8719563be1" containerID="6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af" exitCode=0
Jul 18 01:50:11.779340 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.779298    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf"
Jul 18 01:50:11.779552 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.779296    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf" event={"ID":"ae958a97-679e-4c2b-bb4d-6c8719563be1","Type":"ContainerDied","Data":"6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af"}
Jul 18 01:50:11.779552 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.779434    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-99ff6f44d-sngjf" event={"ID":"ae958a97-679e-4c2b-bb4d-6c8719563be1","Type":"ContainerDied","Data":"ec94e213ba5d7fb1bb663a2d10b2716277bbb2c6ce864c828c5ca9dc0e82cefc"}
Jul 18 01:50:11.779552 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.779463    2552 scope.go:117] "RemoveContainer" containerID="6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af"
Jul 18 01:50:11.788186 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.788169    2552 scope.go:117] "RemoveContainer" containerID="6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af"
Jul 18 01:50:11.788521 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:50:11.788501    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af\": container with ID starting with 6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af not found: ID does not exist" containerID="6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af"
Jul 18 01:50:11.788584 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.788531    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af"} err="failed to get container status \"6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af\": rpc error: code = NotFound desc = could not find container \"6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af\": container with ID starting with 6ba891d4302ad1cbe3e58493ce00f56d8453e82332915883735c9481ef3566af not found: ID does not exist"
Jul 18 01:50:11.798925 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.798895    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-99ff6f44d-sngjf"]
Jul 18 01:50:11.802394 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:11.802367    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-99ff6f44d-sngjf"]
Jul 18 01:50:13.074679 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:13.074648    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae958a97-679e-4c2b-bb4d-6c8719563be1" path="/var/lib/kubelet/pods/ae958a97-679e-4c2b-bb4d-6c8719563be1/volumes"
Jul 18 01:50:15.796028 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:15.795994    2552 generic.go:356] "Generic (PLEG): container finished" podID="1d553bb2-7d34-4365-98b0-7b2b90227719" containerID="73628995b8a151b43568bc2e3ff291306a223eea707a4d1b4be238ee58ad96e4" exitCode=0
Jul 18 01:50:15.796487 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:15.796066    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-hqv46" event={"ID":"1d553bb2-7d34-4365-98b0-7b2b90227719","Type":"ContainerDied","Data":"73628995b8a151b43568bc2e3ff291306a223eea707a4d1b4be238ee58ad96e4"}
Jul 18 01:50:15.796487 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:15.796477    2552 scope.go:117] "RemoveContainer" containerID="73628995b8a151b43568bc2e3ff291306a223eea707a4d1b4be238ee58ad96e4"
Jul 18 01:50:15.797476 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:15.797407    2552 generic.go:356] "Generic (PLEG): container finished" podID="31bfb9b1-86b3-4194-8651-68f48a251535" containerID="61e7512a5ce8107e3329d53fc35fe66c646e80f296ecda3edeed78309201b6d2" exitCode=0
Jul 18 01:50:15.797476 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:15.797463    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2" event={"ID":"31bfb9b1-86b3-4194-8651-68f48a251535","Type":"ContainerDied","Data":"61e7512a5ce8107e3329d53fc35fe66c646e80f296ecda3edeed78309201b6d2"}
Jul 18 01:50:15.797762 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:15.797744    2552 scope.go:117] "RemoveContainer" containerID="61e7512a5ce8107e3329d53fc35fe66c646e80f296ecda3edeed78309201b6d2"
Jul 18 01:50:16.426869 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:16.426843    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c89c86f68-22ccf_93ed0884-1025-4a1c-b573-dfa09a741a37/router/0.log"
Jul 18 01:50:16.432717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:16.432693    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bdbxm_1e601610-f330-493b-adf7-13534056efa2/serve-healthcheck-canary/0.log"
Jul 18 01:50:16.803308 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:16.803203    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-hqv46" event={"ID":"1d553bb2-7d34-4365-98b0-7b2b90227719","Type":"ContainerStarted","Data":"180bcaf1d1fee63e51565397c4eaab8eef62fdaab128112d4bfba3f430b94d03"}
Jul 18 01:50:16.804995 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:16.804975    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7w6l2" event={"ID":"31bfb9b1-86b3-4194-8651-68f48a251535","Type":"ContainerStarted","Data":"facf95ca7b70e3d45b95d6b2626e96990578b747f646fe7bdd3bb6ed7e7307f7"}
Jul 18 01:50:26.448849 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:26.448801    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:26.499120 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:26.499084    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:30.859691 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:30.859655    2552 generic.go:356] "Generic (PLEG): container finished" podID="d06d5fa3-648b-4092-bbe9-09c8c0176e1b" containerID="6e440f214527b7728654ed116bd32d6174ce1700645e93fb4c771f5c1f3ee339" exitCode=0
Jul 18 01:50:30.860069 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:30.859726    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp" event={"ID":"d06d5fa3-648b-4092-bbe9-09c8c0176e1b","Type":"ContainerDied","Data":"6e440f214527b7728654ed116bd32d6174ce1700645e93fb4c771f5c1f3ee339"}
Jul 18 01:50:30.860111 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:30.860076    2552 scope.go:117] "RemoveContainer" containerID="6e440f214527b7728654ed116bd32d6174ce1700645e93fb4c771f5c1f3ee339"
Jul 18 01:50:31.865144 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:31.865111    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4gjbp" event={"ID":"d06d5fa3-648b-4092-bbe9-09c8c0176e1b","Type":"ContainerStarted","Data":"2b068b25355f43aabef3b2ec6bde59770a7de9f369460a3948c3e4bb785af3ae"}
Jul 18 01:50:34.522685 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:34.522630    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:34.595864 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:34.595831    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:34.661521 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:34.661480    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:50:39.632866 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:50:39.632838    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:09.119810 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:09.119778    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:09.448005 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:09.447966    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:17.979998 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:17.979968    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:18.032806 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:18.032771    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:24.323685 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:24.323645    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:24.389522 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:24.389484    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:36.160688 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:36.160653    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:36.215674 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:36.215644    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:40.240290 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:40.240258    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:40.289874 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:40.289841    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:40.341730 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:40.341701    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:40.392334 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:40.392306    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:41.457558 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:41.457516    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:41.553120 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:41.553085    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:42.844244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:42.844208    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:42.936175 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:42.936146    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:46.592894 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:46.592859    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:46.644828 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:46.644797    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:59.084468 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.084391    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:59.094630 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.094603    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs"]
Jul 18 01:51:59.094863 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.094852    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae958a97-679e-4c2b-bb4d-6c8719563be1" containerName="registry"
Jul 18 01:51:59.094910 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.094865    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae958a97-679e-4c2b-bb4d-6c8719563be1" containerName="registry"
Jul 18 01:51:59.094942 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.094917    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae958a97-679e-4c2b-bb4d-6c8719563be1" containerName="registry"
Jul 18 01:51:59.097701 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.097684    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs"
Jul 18 01:51:59.100001 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.099983    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 18 01:51:59.100710 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.100692    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 18 01:51:59.100788 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.100711    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-q8rbq\""
Jul 18 01:51:59.100788 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.100692    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 18 01:51:59.100788 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.100733    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 18 01:51:59.105908 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.105888    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs"]
Jul 18 01:51:59.173934 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.173899    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:59.232801 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.232767    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c97k6\" (UniqueName: \"kubernetes.io/projected/7ad14f49-d0d0-4010-ab19-5116bb8039e1-kube-api-access-c97k6\") pod \"managed-serviceaccount-addon-agent-6898cf5bcb-pcngs\" (UID: \"7ad14f49-d0d0-4010-ab19-5116bb8039e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs"
Jul 18 01:51:59.233001 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.232827    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ad14f49-d0d0-4010-ab19-5116bb8039e1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6898cf5bcb-pcngs\" (UID: \"7ad14f49-d0d0-4010-ab19-5116bb8039e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs"
Jul 18 01:51:59.256797 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.256765    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:51:59.333760 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.333728    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ad14f49-d0d0-4010-ab19-5116bb8039e1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6898cf5bcb-pcngs\" (UID: \"7ad14f49-d0d0-4010-ab19-5116bb8039e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs"
Jul 18 01:51:59.333931 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.333776    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c97k6\" (UniqueName: \"kubernetes.io/projected/7ad14f49-d0d0-4010-ab19-5116bb8039e1-kube-api-access-c97k6\") pod \"managed-serviceaccount-addon-agent-6898cf5bcb-pcngs\" (UID: \"7ad14f49-d0d0-4010-ab19-5116bb8039e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs"
Jul 18 01:51:59.336377 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.336316    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ad14f49-d0d0-4010-ab19-5116bb8039e1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6898cf5bcb-pcngs\" (UID: \"7ad14f49-d0d0-4010-ab19-5116bb8039e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs"
Jul 18 01:51:59.343759 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.343737    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c97k6\" (UniqueName: \"kubernetes.io/projected/7ad14f49-d0d0-4010-ab19-5116bb8039e1-kube-api-access-c97k6\") pod \"managed-serviceaccount-addon-agent-6898cf5bcb-pcngs\" (UID: \"7ad14f49-d0d0-4010-ab19-5116bb8039e1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs"
Jul 18 01:51:59.418038 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.418005    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs"
Jul 18 01:51:59.554836 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:51:59.551921    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs"]
Jul 18 01:51:59.558335 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:51:59.558301    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ad14f49_d0d0_4010_ab19_5116bb8039e1.slice/crio-37d869f56356046aeb31444e4717fc63733ba19f24edfced59a92df349d6a625 WatchSource:0}: Error finding container 37d869f56356046aeb31444e4717fc63733ba19f24edfced59a92df349d6a625: Status 404 returned error can't find the container with id 37d869f56356046aeb31444e4717fc63733ba19f24edfced59a92df349d6a625
Jul 18 01:52:00.187297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:00.187251    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs" event={"ID":"7ad14f49-d0d0-4010-ab19-5116bb8039e1","Type":"ContainerStarted","Data":"37d869f56356046aeb31444e4717fc63733ba19f24edfced59a92df349d6a625"}
Jul 18 01:52:02.196110 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:02.196073    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs" event={"ID":"7ad14f49-d0d0-4010-ab19-5116bb8039e1","Type":"ContainerStarted","Data":"9f1012feac985b8f6ff169de331f2cbf6f485fdfea53c9a3f3d5bf30080ba79a"}
Jul 18 01:52:02.221063 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:02.221015    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6898cf5bcb-pcngs" podStartSLOduration=1.035656756 podStartE2EDuration="3.220996276s" totalImagesPullingTime="2.18533952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:51:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:52:02.220471239 +0000 UTC m=+227.659968171" watchObservedRunningTime="2026-07-18 01:52:02.220996276 +0000 UTC m=+227.660493209"
Jul 18 01:52:22.888828 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:22.888790    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:29.860562 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:29.860521    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:29.959557 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:29.959519    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:30.528051 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.528018    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"]
Jul 18 01:52:30.536544 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.536519    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:30.538815 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.538789    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p8g2l\""
Jul 18 01:52:30.539220 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.539201    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 18 01:52:30.539308 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.539271    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 18 01:52:30.540230 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.540207    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"]
Jul 18 01:52:30.671124 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.671094    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:30.671283 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.671160    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5ftt\" (UniqueName: \"kubernetes.io/projected/333fc41a-d38f-4901-b61a-cdddde7ead64-kube-api-access-l5ftt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:30.671283 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.671213    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:30.772354 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.772320    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5ftt\" (UniqueName: \"kubernetes.io/projected/333fc41a-d38f-4901-b61a-cdddde7ead64-kube-api-access-l5ftt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:30.772354 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.772358    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:30.772565 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.772394    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:30.772794 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.772775    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:30.772834 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.772789    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:30.781545 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.781476    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5ftt\" (UniqueName: \"kubernetes.io/projected/333fc41a-d38f-4901-b61a-cdddde7ead64-kube-api-access-l5ftt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:30.848496 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.848466    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:30.977334 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:30.977159    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"]
Jul 18 01:52:30.980178 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:52:30.980153    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod333fc41a_d38f_4901_b61a_cdddde7ead64.slice/crio-de2bd4bed5db28998ca5b12615406be150feb175cbaf9b0fc35845bfa006276e WatchSource:0}: Error finding container de2bd4bed5db28998ca5b12615406be150feb175cbaf9b0fc35845bfa006276e: Status 404 returned error can't find the container with id de2bd4bed5db28998ca5b12615406be150feb175cbaf9b0fc35845bfa006276e
Jul 18 01:52:31.297744 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:31.297710    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh" event={"ID":"333fc41a-d38f-4901-b61a-cdddde7ead64","Type":"ContainerStarted","Data":"de2bd4bed5db28998ca5b12615406be150feb175cbaf9b0fc35845bfa006276e"}
Jul 18 01:52:32.236392 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:32.236352    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:35.844438 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:35.844324    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:38.326636 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:38.326603    2552 generic.go:356] "Generic (PLEG): container finished" podID="333fc41a-d38f-4901-b61a-cdddde7ead64" containerID="b5f9b3f224ea3fcc59b3b15c637ba7c694bce22d76662f91d20833cecd9d02e4" exitCode=0
Jul 18 01:52:38.327095 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:38.326690    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh" event={"ID":"333fc41a-d38f-4901-b61a-cdddde7ead64","Type":"ContainerDied","Data":"b5f9b3f224ea3fcc59b3b15c637ba7c694bce22d76662f91d20833cecd9d02e4"}
Jul 18 01:52:41.340159 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:41.340125    2552 generic.go:356] "Generic (PLEG): container finished" podID="333fc41a-d38f-4901-b61a-cdddde7ead64" containerID="8c2951c1fd9e175bc02e0840fc2cf967094d05b2864fe3f6bb53afc9cef84da9" exitCode=0
Jul 18 01:52:41.340567 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:41.340176    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh" event={"ID":"333fc41a-d38f-4901-b61a-cdddde7ead64","Type":"ContainerDied","Data":"8c2951c1fd9e175bc02e0840fc2cf967094d05b2864fe3f6bb53afc9cef84da9"}
Jul 18 01:52:48.372322 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:48.372293    2552 generic.go:356] "Generic (PLEG): container finished" podID="333fc41a-d38f-4901-b61a-cdddde7ead64" containerID="760318bb5d863ecdc8462b2cc2273a6603f1cf904b16dfa693a096e33efd1e45" exitCode=0
Jul 18 01:52:48.372695 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:48.372341    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh" event={"ID":"333fc41a-d38f-4901-b61a-cdddde7ead64","Type":"ContainerDied","Data":"760318bb5d863ecdc8462b2cc2273a6603f1cf904b16dfa693a096e33efd1e45"}
Jul 18 01:52:49.503894 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:49.503868    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:49.506723 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:49.506703    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5ftt\" (UniqueName: \"kubernetes.io/projected/333fc41a-d38f-4901-b61a-cdddde7ead64-kube-api-access-l5ftt\") pod \"333fc41a-d38f-4901-b61a-cdddde7ead64\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") "
Jul 18 01:52:49.506782 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:49.506735    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-bundle\") pod \"333fc41a-d38f-4901-b61a-cdddde7ead64\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") "
Jul 18 01:52:49.506782 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:49.506768    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-util\") pod \"333fc41a-d38f-4901-b61a-cdddde7ead64\" (UID: \"333fc41a-d38f-4901-b61a-cdddde7ead64\") "
Jul 18 01:52:49.507456 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:49.507408    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-bundle" (OuterVolumeSpecName: "bundle") pod "333fc41a-d38f-4901-b61a-cdddde7ead64" (UID: "333fc41a-d38f-4901-b61a-cdddde7ead64"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:52:49.509027 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:49.509006    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/333fc41a-d38f-4901-b61a-cdddde7ead64-kube-api-access-l5ftt" (OuterVolumeSpecName: "kube-api-access-l5ftt") pod "333fc41a-d38f-4901-b61a-cdddde7ead64" (UID: "333fc41a-d38f-4901-b61a-cdddde7ead64"). InnerVolumeSpecName "kube-api-access-l5ftt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:52:49.511964 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:49.511938    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-util" (OuterVolumeSpecName: "util") pod "333fc41a-d38f-4901-b61a-cdddde7ead64" (UID: "333fc41a-d38f-4901-b61a-cdddde7ead64"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:52:49.607362 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:49.607321    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5ftt\" (UniqueName: \"kubernetes.io/projected/333fc41a-d38f-4901-b61a-cdddde7ead64-kube-api-access-l5ftt\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:52:49.607362 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:49.607359    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-bundle\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:52:49.607362 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:49.607369    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/333fc41a-d38f-4901-b61a-cdddde7ead64-util\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:52:50.380939 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:50.380908    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh"
Jul 18 01:52:50.380939 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:50.380920    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404g6gh" event={"ID":"333fc41a-d38f-4901-b61a-cdddde7ead64","Type":"ContainerDied","Data":"de2bd4bed5db28998ca5b12615406be150feb175cbaf9b0fc35845bfa006276e"}
Jul 18 01:52:50.381145 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:50.380960    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de2bd4bed5db28998ca5b12615406be150feb175cbaf9b0fc35845bfa006276e"
Jul 18 01:52:51.536652 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:51.536621    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:52.366898 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:52.366865    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:53.507409 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.507375    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg"]
Jul 18 01:52:53.507794 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.507656    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="333fc41a-d38f-4901-b61a-cdddde7ead64" containerName="util"
Jul 18 01:52:53.507794 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.507667    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="333fc41a-d38f-4901-b61a-cdddde7ead64" containerName="util"
Jul 18 01:52:53.507794 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.507675    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="333fc41a-d38f-4901-b61a-cdddde7ead64" containerName="pull"
Jul 18 01:52:53.507794 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.507680    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="333fc41a-d38f-4901-b61a-cdddde7ead64" containerName="pull"
Jul 18 01:52:53.507794 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.507688    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="333fc41a-d38f-4901-b61a-cdddde7ead64" containerName="extract"
Jul 18 01:52:53.507794 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.507693    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="333fc41a-d38f-4901-b61a-cdddde7ead64" containerName="extract"
Jul 18 01:52:53.507794 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.507745    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="333fc41a-d38f-4901-b61a-cdddde7ead64" containerName="extract"
Jul 18 01:52:53.550562 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.550529    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg"]
Jul 18 01:52:53.550717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.550623    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg"
Jul 18 01:52:53.552775 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.552754    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 18 01:52:53.552886 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.552755    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:52:53.552941 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.552923    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-b8drj\""
Jul 18 01:52:53.638387 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.638349    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v8jbg\" (UID: \"d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg"
Jul 18 01:52:53.638573 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.638395    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wg5v\" (UniqueName: \"kubernetes.io/projected/d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030-kube-api-access-4wg5v\") pod \"cert-manager-operator-controller-manager-55c444bf86-v8jbg\" (UID: \"d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg"
Jul 18 01:52:53.739066 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.739031    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v8jbg\" (UID: \"d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg"
Jul 18 01:52:53.739066 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.739076    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wg5v\" (UniqueName: \"kubernetes.io/projected/d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030-kube-api-access-4wg5v\") pod \"cert-manager-operator-controller-manager-55c444bf86-v8jbg\" (UID: \"d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg"
Jul 18 01:52:53.739474 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.739451    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v8jbg\" (UID: \"d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg"
Jul 18 01:52:53.755132 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.755111    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wg5v\" (UniqueName: \"kubernetes.io/projected/d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030-kube-api-access-4wg5v\") pod \"cert-manager-operator-controller-manager-55c444bf86-v8jbg\" (UID: \"d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg"
Jul 18 01:52:53.859866 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:53.859776    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg"
Jul 18 01:52:54.021576 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:54.021545    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg"]
Jul 18 01:52:54.025503 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:52:54.025474    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29a6b40_fe8a_4266_bdd9_ae8c4a0ed030.slice/crio-3251b67fe71e767590313ce241d3708e38ae7dc250fc38f2ba46165d078c4f0d WatchSource:0}: Error finding container 3251b67fe71e767590313ce241d3708e38ae7dc250fc38f2ba46165d078c4f0d: Status 404 returned error can't find the container with id 3251b67fe71e767590313ce241d3708e38ae7dc250fc38f2ba46165d078c4f0d
Jul 18 01:52:54.397819 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:54.397787    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg" event={"ID":"d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030","Type":"ContainerStarted","Data":"3251b67fe71e767590313ce241d3708e38ae7dc250fc38f2ba46165d078c4f0d"}
Jul 18 01:52:55.392016 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:55.391976    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:55.491370 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:55.491340    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:56.408246 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:56.408207    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg" event={"ID":"d29a6b40-fe8a-4266-bdd9-ae8c4a0ed030","Type":"ContainerStarted","Data":"3f96d406498a93fa7344f44c54e0a57c6ea2a5a2ebfbb7f96e31fa7859c3cdc6"}
Jul 18 01:52:56.448946 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:56.448886    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v8jbg" podStartSLOduration=1.256114907 podStartE2EDuration="3.448868286s" totalImagesPullingTime="2.192753379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:52:56.447890897 +0000 UTC m=+281.887387834" watchObservedRunningTime="2026-07-18 01:52:56.448868286 +0000 UTC m=+281.888365218"
Jul 18 01:52:56.850513 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:56.850456    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:56.980973 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:56.980932    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:57.189325 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.189287    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:57.346910 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.346881    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:57.671944 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.671907    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-c9jrr"]
Jul 18 01:52:57.674204 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.674177    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"]
Jul 18 01:52:57.674359 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.674281    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr"
Jul 18 01:52:57.676342 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.676318    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 18 01:52:57.676472 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.676385    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 18 01:52:57.676712 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.676694    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:52:57.681184 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.681159    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 18 01:52:57.681359 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.681341    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p8g2l\""
Jul 18 01:52:57.681538 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.681521    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 18 01:52:57.689350 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.689328    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-c9jrr"]
Jul 18 01:52:57.696453 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.696400    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"]
Jul 18 01:52:57.772632 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.772596    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvcrm\" (UniqueName: \"kubernetes.io/projected/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-kube-api-access-qvcrm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:52:57.772811 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.772638    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fa94f0d-e10c-4e1c-96bd-6aefabc730c5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-c9jrr\" (UID: \"2fa94f0d-e10c-4e1c-96bd-6aefabc730c5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr"
Jul 18 01:52:57.772811 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.772715    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:52:57.772811 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.772763    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tftmp\" (UniqueName: \"kubernetes.io/projected/2fa94f0d-e10c-4e1c-96bd-6aefabc730c5-kube-api-access-tftmp\") pod \"cert-manager-webhook-6486ffd997-c9jrr\" (UID: \"2fa94f0d-e10c-4e1c-96bd-6aefabc730c5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr"
Jul 18 01:52:57.772811 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.772780    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:52:57.873510 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.873476    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:52:57.873687 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.873543    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvcrm\" (UniqueName: \"kubernetes.io/projected/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-kube-api-access-qvcrm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:52:57.873687 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.873568    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fa94f0d-e10c-4e1c-96bd-6aefabc730c5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-c9jrr\" (UID: \"2fa94f0d-e10c-4e1c-96bd-6aefabc730c5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr"
Jul 18 01:52:57.873687 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.873593    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:52:57.873687 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.873619    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tftmp\" (UniqueName: \"kubernetes.io/projected/2fa94f0d-e10c-4e1c-96bd-6aefabc730c5-kube-api-access-tftmp\") pod \"cert-manager-webhook-6486ffd997-c9jrr\" (UID: \"2fa94f0d-e10c-4e1c-96bd-6aefabc730c5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr"
Jul 18 01:52:57.873895 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.873862    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:52:57.873979 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.873957    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:52:57.881896 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.881867    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fa94f0d-e10c-4e1c-96bd-6aefabc730c5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-c9jrr\" (UID: \"2fa94f0d-e10c-4e1c-96bd-6aefabc730c5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr"
Jul 18 01:52:57.882244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.882220    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tftmp\" (UniqueName: \"kubernetes.io/projected/2fa94f0d-e10c-4e1c-96bd-6aefabc730c5-kube-api-access-tftmp\") pod \"cert-manager-webhook-6486ffd997-c9jrr\" (UID: \"2fa94f0d-e10c-4e1c-96bd-6aefabc730c5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr"
Jul 18 01:52:57.882407 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.882387    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvcrm\" (UniqueName: \"kubernetes.io/projected/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-kube-api-access-qvcrm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:52:57.984392 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.984303    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr"
Jul 18 01:52:57.990218 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:57.990194    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:52:58.125589 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:58.125559    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-c9jrr"]
Jul 18 01:52:58.129073 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:52:58.129039    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fa94f0d_e10c_4e1c_96bd_6aefabc730c5.slice/crio-9ba9bbf6a25bd6fb1e94f93d894220675a309f3b6315a6fd7d40be8443d80c98 WatchSource:0}: Error finding container 9ba9bbf6a25bd6fb1e94f93d894220675a309f3b6315a6fd7d40be8443d80c98: Status 404 returned error can't find the container with id 9ba9bbf6a25bd6fb1e94f93d894220675a309f3b6315a6fd7d40be8443d80c98
Jul 18 01:52:58.149757 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:58.149730    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"]
Jul 18 01:52:58.152001 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:52:58.151976    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b3e24fe_2fd3_42fc_a4a0_706b0fd0f85d.slice/crio-60e7b38cc89c3e32776310d1986aecd40ecb7a955fd979d0d4ec0c6110eb2dc6 WatchSource:0}: Error finding container 60e7b38cc89c3e32776310d1986aecd40ecb7a955fd979d0d4ec0c6110eb2dc6: Status 404 returned error can't find the container with id 60e7b38cc89c3e32776310d1986aecd40ecb7a955fd979d0d4ec0c6110eb2dc6
Jul 18 01:52:58.418790 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:58.418753    2552 generic.go:356] "Generic (PLEG): container finished" podID="0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" containerID="8e24210f9e042b8e0e74070487f6dbea0eaba59fddce53ddc65b9d16705ab454" exitCode=0
Jul 18 01:52:58.418966 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:58.418837    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr" event={"ID":"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d","Type":"ContainerDied","Data":"8e24210f9e042b8e0e74070487f6dbea0eaba59fddce53ddc65b9d16705ab454"}
Jul 18 01:52:58.418966 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:58.418887    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr" event={"ID":"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d","Type":"ContainerStarted","Data":"60e7b38cc89c3e32776310d1986aecd40ecb7a955fd979d0d4ec0c6110eb2dc6"}
Jul 18 01:52:58.420245 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:58.420216    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr" event={"ID":"2fa94f0d-e10c-4e1c-96bd-6aefabc730c5","Type":"ContainerStarted","Data":"9ba9bbf6a25bd6fb1e94f93d894220675a309f3b6315a6fd7d40be8443d80c98"}
Jul 18 01:52:59.350941 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:59.350902    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:52:59.964015 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:52:59.963981    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:01.437297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:01.437262    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr" event={"ID":"2fa94f0d-e10c-4e1c-96bd-6aefabc730c5","Type":"ContainerStarted","Data":"1e13ab78146366b2442597e8245cb42aa4e03bab60a21a27c94ceee7eb24f651"}
Jul 18 01:53:01.437652 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:01.437347    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr"
Jul 18 01:53:01.439391 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:01.439370    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr" event={"ID":"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d","Type":"ContainerStarted","Data":"665c73969fa0e67b0a37b8456a37fa33daf48d6486baead58af4d2102b4be593"}
Jul 18 01:53:01.454838 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:01.454799    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr" podStartSLOduration=1.2874280439999999 podStartE2EDuration="4.454785387s" totalImagesPullingTime="3.167357343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:52:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:01.454123916 +0000 UTC m=+286.893620859" watchObservedRunningTime="2026-07-18 01:53:01.454785387 +0000 UTC m=+286.894282319"
Jul 18 01:53:01.946926 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:01.946897    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:02.012089 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:02.012061    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:02.444726 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:02.444699    2552 generic.go:356] "Generic (PLEG): container finished" podID="0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" containerID="665c73969fa0e67b0a37b8456a37fa33daf48d6486baead58af4d2102b4be593" exitCode=0
Jul 18 01:53:02.445113 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:02.444787    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr" event={"ID":"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d","Type":"ContainerDied","Data":"665c73969fa0e67b0a37b8456a37fa33daf48d6486baead58af4d2102b4be593"}
Jul 18 01:53:03.450747 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:03.450709    2552 generic.go:356] "Generic (PLEG): container finished" podID="0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" containerID="44c94792a423df8520fd656c04bfd3e5dec9a979476d28c6b3f9ab402b751ca9" exitCode=0
Jul 18 01:53:03.451132 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:03.450773    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr" event={"ID":"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d","Type":"ContainerDied","Data":"44c94792a423df8520fd656c04bfd3e5dec9a979476d28c6b3f9ab402b751ca9"}
Jul 18 01:53:04.582003 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:04.581975    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:53:04.630312 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:04.630276    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-bundle\") pod \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") "
Jul 18 01:53:04.630497 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:04.630326    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvcrm\" (UniqueName: \"kubernetes.io/projected/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-kube-api-access-qvcrm\") pod \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") "
Jul 18 01:53:04.630497 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:04.630348    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-util\") pod \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\" (UID: \"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d\") "
Jul 18 01:53:04.630795 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:04.630759    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-bundle" (OuterVolumeSpecName: "bundle") pod "0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" (UID: "0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:53:04.632703 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:04.632680    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-kube-api-access-qvcrm" (OuterVolumeSpecName: "kube-api-access-qvcrm") pod "0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" (UID: "0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d"). InnerVolumeSpecName "kube-api-access-qvcrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:53:04.634300 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:04.634281    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-util" (OuterVolumeSpecName: "util") pod "0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" (UID: "0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:53:04.731485 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:04.731367    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-bundle\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:53:04.731485 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:04.731402    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvcrm\" (UniqueName: \"kubernetes.io/projected/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-kube-api-access-qvcrm\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:53:04.731485 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:04.731413    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d-util\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:53:05.087540 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:05.087511    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:05.291084 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:05.291046    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:05.460713 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:05.460680    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr" event={"ID":"0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d","Type":"ContainerDied","Data":"60e7b38cc89c3e32776310d1986aecd40ecb7a955fd979d0d4ec0c6110eb2dc6"}
Jul 18 01:53:05.460912 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:05.460733    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60e7b38cc89c3e32776310d1986aecd40ecb7a955fd979d0d4ec0c6110eb2dc6"
Jul 18 01:53:05.460912 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:05.460692    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2cnvr"
Jul 18 01:53:07.447479 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:07.447452    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-c9jrr"
Jul 18 01:53:07.770333 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:07.770248    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:09.311856 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:09.311824    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:09.512830 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:09.512793    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:14.946218 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:14.946186    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 01:53:14.947969 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:14.947663    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 01:53:14.956930 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:14.956906    2552 kubelet.go:1648] "Image garbage collection succeeded"
Jul 18 01:53:15.238546 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:15.238467    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:16.539672 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:16.539636    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:17.026904 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.026863    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:17.191070 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.191029    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"]
Jul 18 01:53:17.191301 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.191290    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" containerName="extract"
Jul 18 01:53:17.191343 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.191304    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" containerName="extract"
Jul 18 01:53:17.191343 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.191319    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" containerName="pull"
Jul 18 01:53:17.191343 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.191325    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" containerName="pull"
Jul 18 01:53:17.191343 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.191336    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" containerName="util"
Jul 18 01:53:17.191343 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.191341    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" containerName="util"
Jul 18 01:53:17.191518 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.191393    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b3e24fe-2fd3-42fc-a4a0-706b0fd0f85d" containerName="extract"
Jul 18 01:53:17.194476 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.194458    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:17.196577 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.196557    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p8g2l\""
Jul 18 01:53:17.196875 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.196864    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 18 01:53:17.197220 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.197203    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 18 01:53:17.202833 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.202809    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"]
Jul 18 01:53:17.225644 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.225601    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c4zj\" (UniqueName: \"kubernetes.io/projected/5ad91b08-773f-4703-b0fa-f199294473f0-kube-api-access-4c4zj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:17.225816 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.225659    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:17.225816 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.225681    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:17.327021 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.326926    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c4zj\" (UniqueName: \"kubernetes.io/projected/5ad91b08-773f-4703-b0fa-f199294473f0-kube-api-access-4c4zj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:17.327021 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.326972    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:17.327021 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.326990    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:17.327316 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.327299    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:17.327353 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.327325    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:17.335097 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.335070    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c4zj\" (UniqueName: \"kubernetes.io/projected/5ad91b08-773f-4703-b0fa-f199294473f0-kube-api-access-4c4zj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:17.504204 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.504163    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:17.647294 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:17.647253    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"]
Jul 18 01:53:17.652998 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:53:17.652961    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad91b08_773f_4703_b0fa_f199294473f0.slice/crio-0665d5dc48139bafec2665e1d460a88b7f5e23a7426de8ac00dc10d17e5dd453 WatchSource:0}: Error finding container 0665d5dc48139bafec2665e1d460a88b7f5e23a7426de8ac00dc10d17e5dd453: Status 404 returned error can't find the container with id 0665d5dc48139bafec2665e1d460a88b7f5e23a7426de8ac00dc10d17e5dd453
Jul 18 01:53:18.519411 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:18.519376    2552 generic.go:356] "Generic (PLEG): container finished" podID="5ad91b08-773f-4703-b0fa-f199294473f0" containerID="888fdfc0a74be34fc9a3f973c15057657548df85106db251777678897ceccf78" exitCode=0
Jul 18 01:53:18.519618 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:18.519471    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4" event={"ID":"5ad91b08-773f-4703-b0fa-f199294473f0","Type":"ContainerDied","Data":"888fdfc0a74be34fc9a3f973c15057657548df85106db251777678897ceccf78"}
Jul 18 01:53:18.519618 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:18.519519    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4" event={"ID":"5ad91b08-773f-4703-b0fa-f199294473f0","Type":"ContainerStarted","Data":"0665d5dc48139bafec2665e1d460a88b7f5e23a7426de8ac00dc10d17e5dd453"}
Jul 18 01:53:18.519934 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:18.519920    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:53:19.525309 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:19.525271    2552 generic.go:356] "Generic (PLEG): container finished" podID="5ad91b08-773f-4703-b0fa-f199294473f0" containerID="046e554ec3f047170e047c6838ef3bf67fe3bed3cc62da1a84af3c480897f2a0" exitCode=0
Jul 18 01:53:19.525759 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:19.525360    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4" event={"ID":"5ad91b08-773f-4703-b0fa-f199294473f0","Type":"ContainerDied","Data":"046e554ec3f047170e047c6838ef3bf67fe3bed3cc62da1a84af3c480897f2a0"}
Jul 18 01:53:20.531339 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:20.531302    2552 generic.go:356] "Generic (PLEG): container finished" podID="5ad91b08-773f-4703-b0fa-f199294473f0" containerID="5ad4ca1ae4da201034c8d8398ba2a5c6d5a6dae35722fa5ce39288184ab94ec1" exitCode=0
Jul 18 01:53:20.531806 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:20.531382    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4" event={"ID":"5ad91b08-773f-4703-b0fa-f199294473f0","Type":"ContainerDied","Data":"5ad4ca1ae4da201034c8d8398ba2a5c6d5a6dae35722fa5ce39288184ab94ec1"}
Jul 18 01:53:21.672337 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:21.672313    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:21.756942 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:21.756906    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c4zj\" (UniqueName: \"kubernetes.io/projected/5ad91b08-773f-4703-b0fa-f199294473f0-kube-api-access-4c4zj\") pod \"5ad91b08-773f-4703-b0fa-f199294473f0\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") "
Jul 18 01:53:21.757112 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:21.756958    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-util\") pod \"5ad91b08-773f-4703-b0fa-f199294473f0\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") "
Jul 18 01:53:21.757112 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:21.756991    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-bundle\") pod \"5ad91b08-773f-4703-b0fa-f199294473f0\" (UID: \"5ad91b08-773f-4703-b0fa-f199294473f0\") "
Jul 18 01:53:21.757933 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:21.757886    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-bundle" (OuterVolumeSpecName: "bundle") pod "5ad91b08-773f-4703-b0fa-f199294473f0" (UID: "5ad91b08-773f-4703-b0fa-f199294473f0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:53:21.759108 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:21.759082    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ad91b08-773f-4703-b0fa-f199294473f0-kube-api-access-4c4zj" (OuterVolumeSpecName: "kube-api-access-4c4zj") pod "5ad91b08-773f-4703-b0fa-f199294473f0" (UID: "5ad91b08-773f-4703-b0fa-f199294473f0"). InnerVolumeSpecName "kube-api-access-4c4zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:53:21.763119 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:21.763100    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-util" (OuterVolumeSpecName: "util") pod "5ad91b08-773f-4703-b0fa-f199294473f0" (UID: "5ad91b08-773f-4703-b0fa-f199294473f0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:53:21.857723 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:21.857620    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c4zj\" (UniqueName: \"kubernetes.io/projected/5ad91b08-773f-4703-b0fa-f199294473f0-kube-api-access-4c4zj\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:53:21.857723 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:21.857667    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-util\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:53:21.857723 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:21.857678    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ad91b08-773f-4703-b0fa-f199294473f0-bundle\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:53:22.542002 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:22.541977    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4"
Jul 18 01:53:22.542172 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:22.541973    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgbzl4" event={"ID":"5ad91b08-773f-4703-b0fa-f199294473f0","Type":"ContainerDied","Data":"0665d5dc48139bafec2665e1d460a88b7f5e23a7426de8ac00dc10d17e5dd453"}
Jul 18 01:53:22.542172 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:22.542117    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0665d5dc48139bafec2665e1d460a88b7f5e23a7426de8ac00dc10d17e5dd453"
Jul 18 01:53:29.784151 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:29.784113    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:29.898962 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:29.898925    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:30.411160 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.411114    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:30.565813 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.565784    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:30.664667 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.664579    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:30.703285 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.703256    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"]
Jul 18 01:53:30.703589 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.703577    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad91b08-773f-4703-b0fa-f199294473f0" containerName="util"
Jul 18 01:53:30.703634 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.703592    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad91b08-773f-4703-b0fa-f199294473f0" containerName="util"
Jul 18 01:53:30.703634 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.703600    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad91b08-773f-4703-b0fa-f199294473f0" containerName="extract"
Jul 18 01:53:30.703634 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.703605    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad91b08-773f-4703-b0fa-f199294473f0" containerName="extract"
Jul 18 01:53:30.703634 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.703614    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad91b08-773f-4703-b0fa-f199294473f0" containerName="pull"
Jul 18 01:53:30.703634 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.703620    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad91b08-773f-4703-b0fa-f199294473f0" containerName="pull"
Jul 18 01:53:30.703778 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.703681    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ad91b08-773f-4703-b0fa-f199294473f0" containerName="extract"
Jul 18 01:53:30.733304 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.733276    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.736064 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.736039    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"]
Jul 18 01:53:30.736331 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.736319    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 18 01:53:30.737251 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.737235    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 18 01:53:30.737337 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.737323    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 18 01:53:30.737495 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.737478    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 18 01:53:30.737596 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.737479    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8n9fz\""
Jul 18 01:53:30.830466 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.830414    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-manifests\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.830835 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.830474    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-tmp\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.830835 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.830537    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.830835 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.830582    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.830835 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.830631    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f47vh\" (UniqueName: \"kubernetes.io/projected/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-kube-api-access-f47vh\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.923183 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.923103    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:30.931290 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.931265    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.931461 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.931315    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f47vh\" (UniqueName: \"kubernetes.io/projected/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-kube-api-access-f47vh\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.931461 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.931362    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-manifests\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.931461 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.931397    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-tmp\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.931640 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.931451    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.931913 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.931893    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-manifests\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.933687 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.933665    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-tmp\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.933982 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.933965    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.934031 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.934010    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:30.940270 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:30.940250    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f47vh\" (UniqueName: \"kubernetes.io/projected/1e5c1c18-3295-4685-b91a-ecbd7c97f4fa-kube-api-access-f47vh\") pod \"opendatahub-operator-controller-manager-6b7456dd48-52dcc\" (UID: \"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:31.037578 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:31.037546    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:31.043532 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:31.043506    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:31.165564 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:31.165525    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:31.213106 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:31.213035    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"]
Jul 18 01:53:31.216188 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:53:31.216152    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5c1c18_3295_4685_b91a_ecbd7c97f4fa.slice/crio-5db8ef11d8f10c153979ba28cd83f89a276998f99ca98aa13756c3e75a2e3269 WatchSource:0}: Error finding container 5db8ef11d8f10c153979ba28cd83f89a276998f99ca98aa13756c3e75a2e3269: Status 404 returned error can't find the container with id 5db8ef11d8f10c153979ba28cd83f89a276998f99ca98aa13756c3e75a2e3269
Jul 18 01:53:31.267805 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:31.267772    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:31.370235 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:31.370198    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:31.471688 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:31.471605    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:31.588183 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:31.588153    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc" event={"ID":"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa","Type":"ContainerStarted","Data":"5db8ef11d8f10c153979ba28cd83f89a276998f99ca98aa13756c3e75a2e3269"}
Jul 18 01:53:34.605286 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:34.605250    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc" event={"ID":"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa","Type":"ContainerStarted","Data":"95d2e91367cdff8fde914f4569cca5309a571c40f9cf197eb3fedd30d19f1483"}
Jul 18 01:53:35.611279 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:35.611247    2552 generic.go:356] "Generic (PLEG): container finished" podID="1e5c1c18-3295-4685-b91a-ecbd7c97f4fa" containerID="95d2e91367cdff8fde914f4569cca5309a571c40f9cf197eb3fedd30d19f1483" exitCode=0
Jul 18 01:53:35.611716 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:35.611335    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc" event={"ID":"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa","Type":"ContainerDied","Data":"95d2e91367cdff8fde914f4569cca5309a571c40f9cf197eb3fedd30d19f1483"}
Jul 18 01:53:36.618099 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:36.618058    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc" event={"ID":"1e5c1c18-3295-4685-b91a-ecbd7c97f4fa","Type":"ContainerStarted","Data":"27abb2bc1cb2c91401f6856c095191495b28e47118fe5e086f46e3e11efc5ae4"}
Jul 18 01:53:36.618601 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:36.618287    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:36.638050 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:36.638000    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc" podStartSLOduration=2.586440611 podStartE2EDuration="6.637982959s" totalImagesPullingTime="3.037918571s" totalInitContainerRuntime="1.013623777s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:36.636596626 +0000 UTC m=+322.076093591" watchObservedRunningTime="2026-07-18 01:53:36.637982959 +0000 UTC m=+322.077479894"
Jul 18 01:53:37.283130 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:37.283104    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:37.347352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:37.347321    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:41.935738 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:41.935707    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:42.036090 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:42.036058    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:47.625337 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:47.625308    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b7456dd48-52dcc"
Jul 18 01:53:49.677640 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:49.677587    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:49.826470 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:49.826409    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:50.390902 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.390868    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"]
Jul 18 01:53:50.393368 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.393340    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:50.399706 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.399687    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"
Jul 18 01:53:50.402322 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.402298    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 18 01:53:50.402413 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.402344    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 18 01:53:50.424398 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.424362    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"]
Jul 18 01:53:50.489358 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.489319    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bfa6dcf-1126-4847-a901-92d8b90d866f-tmp\") pod \"kube-auth-proxy-58f5976ff9-lftlr\" (UID: \"0bfa6dcf-1126-4847-a901-92d8b90d866f\") " pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"
Jul 18 01:53:50.489549 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.489389    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bfa6dcf-1126-4847-a901-92d8b90d866f-tls-certs\") pod \"kube-auth-proxy-58f5976ff9-lftlr\" (UID: \"0bfa6dcf-1126-4847-a901-92d8b90d866f\") " pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"
Jul 18 01:53:50.489549 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.489435    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctg4q\" (UniqueName: \"kubernetes.io/projected/0bfa6dcf-1126-4847-a901-92d8b90d866f-kube-api-access-ctg4q\") pod \"kube-auth-proxy-58f5976ff9-lftlr\" (UID: \"0bfa6dcf-1126-4847-a901-92d8b90d866f\") " pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"
Jul 18 01:53:50.544109 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.544070    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qcsz8"]
Jul 18 01:53:50.547715 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.547694    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8"
Jul 18 01:53:50.552173 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.552153    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tpw2d\""
Jul 18 01:53:50.552490 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.552474    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 18 01:53:50.565617 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.565590    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qcsz8"]
Jul 18 01:53:50.589774 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.589753    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bfa6dcf-1126-4847-a901-92d8b90d866f-tls-certs\") pod \"kube-auth-proxy-58f5976ff9-lftlr\" (UID: \"0bfa6dcf-1126-4847-a901-92d8b90d866f\") " pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"
Jul 18 01:53:50.589892 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.589789    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctg4q\" (UniqueName: \"kubernetes.io/projected/0bfa6dcf-1126-4847-a901-92d8b90d866f-kube-api-access-ctg4q\") pod \"kube-auth-proxy-58f5976ff9-lftlr\" (UID: \"0bfa6dcf-1126-4847-a901-92d8b90d866f\") " pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"
Jul 18 01:53:50.589892 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.589840    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bfa6dcf-1126-4847-a901-92d8b90d866f-tmp\") pod \"kube-auth-proxy-58f5976ff9-lftlr\" (UID: \"0bfa6dcf-1126-4847-a901-92d8b90d866f\") " pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"
Jul 18 01:53:50.592237 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.592209    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bfa6dcf-1126-4847-a901-92d8b90d866f-tmp\") pod \"kube-auth-proxy-58f5976ff9-lftlr\" (UID: \"0bfa6dcf-1126-4847-a901-92d8b90d866f\") " pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"
Jul 18 01:53:50.592476 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.592458    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bfa6dcf-1126-4847-a901-92d8b90d866f-tls-certs\") pod \"kube-auth-proxy-58f5976ff9-lftlr\" (UID: \"0bfa6dcf-1126-4847-a901-92d8b90d866f\") " pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"
Jul 18 01:53:50.600303 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.600283    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:50.603171 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.603154    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctg4q\" (UniqueName: \"kubernetes.io/projected/0bfa6dcf-1126-4847-a901-92d8b90d866f-kube-api-access-ctg4q\") pod \"kube-auth-proxy-58f5976ff9-lftlr\" (UID: \"0bfa6dcf-1126-4847-a901-92d8b90d866f\") " pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"
Jul 18 01:53:50.691084 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.690995    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e37d7d55-2e3b-4cd2-8c2a-fe5078acd475-cert\") pod \"odh-model-controller-7998c67dc4-qcsz8\" (UID: \"e37d7d55-2e3b-4cd2-8c2a-fe5078acd475\") " pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8"
Jul 18 01:53:50.691084 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.691043    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxdqr\" (UniqueName: \"kubernetes.io/projected/e37d7d55-2e3b-4cd2-8c2a-fe5078acd475-kube-api-access-hxdqr\") pod \"odh-model-controller-7998c67dc4-qcsz8\" (UID: \"e37d7d55-2e3b-4cd2-8c2a-fe5078acd475\") " pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8"
Jul 18 01:53:50.692228 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.692202    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:50.709152 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.709119    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"
Jul 18 01:53:50.791776 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.791741    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e37d7d55-2e3b-4cd2-8c2a-fe5078acd475-cert\") pod \"odh-model-controller-7998c67dc4-qcsz8\" (UID: \"e37d7d55-2e3b-4cd2-8c2a-fe5078acd475\") " pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8"
Jul 18 01:53:50.791928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.791778    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxdqr\" (UniqueName: \"kubernetes.io/projected/e37d7d55-2e3b-4cd2-8c2a-fe5078acd475-kube-api-access-hxdqr\") pod \"odh-model-controller-7998c67dc4-qcsz8\" (UID: \"e37d7d55-2e3b-4cd2-8c2a-fe5078acd475\") " pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8"
Jul 18 01:53:50.791928 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:53:50.791886    2552 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 18 01:53:50.792006 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:53:50.791951    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e37d7d55-2e3b-4cd2-8c2a-fe5078acd475-cert podName:e37d7d55-2e3b-4cd2-8c2a-fe5078acd475 nodeName:}" failed. No retries permitted until 2026-07-18 01:53:51.291932625 +0000 UTC m=+336.731429552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e37d7d55-2e3b-4cd2-8c2a-fe5078acd475-cert") pod "odh-model-controller-7998c67dc4-qcsz8" (UID: "e37d7d55-2e3b-4cd2-8c2a-fe5078acd475") : secret "odh-model-controller-webhook-cert" not found
Jul 18 01:53:50.792272 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.792248    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:50.800789 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.800762    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxdqr\" (UniqueName: \"kubernetes.io/projected/e37d7d55-2e3b-4cd2-8c2a-fe5078acd475-kube-api-access-hxdqr\") pod \"odh-model-controller-7998c67dc4-qcsz8\" (UID: \"e37d7d55-2e3b-4cd2-8c2a-fe5078acd475\") " pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8"
Jul 18 01:53:50.854386 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.854320    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr"]
Jul 18 01:53:50.857634 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:53:50.857609    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bfa6dcf_1126_4847_a901_92d8b90d866f.slice/crio-e48d7fd007a5015b5f62eec0664a605a17ad0a58d92058f1c1166d918dd4bed3 WatchSource:0}: Error finding container e48d7fd007a5015b5f62eec0664a605a17ad0a58d92058f1c1166d918dd4bed3: Status 404 returned error can't find the container with id e48d7fd007a5015b5f62eec0664a605a17ad0a58d92058f1c1166d918dd4bed3
Jul 18 01:53:50.897388 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.897363    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:50.993453 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:50.993356    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:51.296230 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.296139    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e37d7d55-2e3b-4cd2-8c2a-fe5078acd475-cert\") pod \"odh-model-controller-7998c67dc4-qcsz8\" (UID: \"e37d7d55-2e3b-4cd2-8c2a-fe5078acd475\") " pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8"
Jul 18 01:53:51.298754 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.298726    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e37d7d55-2e3b-4cd2-8c2a-fe5078acd475-cert\") pod \"odh-model-controller-7998c67dc4-qcsz8\" (UID: \"e37d7d55-2e3b-4cd2-8c2a-fe5078acd475\") " pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8"
Jul 18 01:53:51.446440 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.446400    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:51.458286 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.458252    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8"
Jul 18 01:53:51.480804 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.480767    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-szsqv"]
Jul 18 01:53:51.484638 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.484616    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-szsqv"
Jul 18 01:53:51.486949 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.486930    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 18 01:53:51.487061 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.486999    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-bmhxh\""
Jul 18 01:53:51.495842 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.495816    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-szsqv"]
Jul 18 01:53:51.599171 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.599138    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-457nt\" (UniqueName: \"kubernetes.io/projected/7e3b4a90-43a1-452c-b74a-1e77fc21a067-kube-api-access-457nt\") pod \"model-serving-api-54d89bd5fc-szsqv\" (UID: \"7e3b4a90-43a1-452c-b74a-1e77fc21a067\") " pod="opendatahub/model-serving-api-54d89bd5fc-szsqv"
Jul 18 01:53:51.599336 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.599184    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e3b4a90-43a1-452c-b74a-1e77fc21a067-tls-certs\") pod \"model-serving-api-54d89bd5fc-szsqv\" (UID: \"7e3b4a90-43a1-452c-b74a-1e77fc21a067\") " pod="opendatahub/model-serving-api-54d89bd5fc-szsqv"
Jul 18 01:53:51.612658 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.612622    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qcsz8"]
Jul 18 01:53:51.616689 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:53:51.616661    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37d7d55_2e3b_4cd2_8c2a_fe5078acd475.slice/crio-556e6ea99e05744c90a891c86a8ba0c788f5165b234e568e2757b27a0f44e2a4 WatchSource:0}: Error finding container 556e6ea99e05744c90a891c86a8ba0c788f5165b234e568e2757b27a0f44e2a4: Status 404 returned error can't find the container with id 556e6ea99e05744c90a891c86a8ba0c788f5165b234e568e2757b27a0f44e2a4
Jul 18 01:53:51.694485 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.694448    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr" event={"ID":"0bfa6dcf-1126-4847-a901-92d8b90d866f","Type":"ContainerStarted","Data":"e48d7fd007a5015b5f62eec0664a605a17ad0a58d92058f1c1166d918dd4bed3"}
Jul 18 01:53:51.696025 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.695993    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8" event={"ID":"e37d7d55-2e3b-4cd2-8c2a-fe5078acd475","Type":"ContainerStarted","Data":"556e6ea99e05744c90a891c86a8ba0c788f5165b234e568e2757b27a0f44e2a4"}
Jul 18 01:53:51.696993 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.696971    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"]
Jul 18 01:53:51.700491 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.700461    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-457nt\" (UniqueName: \"kubernetes.io/projected/7e3b4a90-43a1-452c-b74a-1e77fc21a067-kube-api-access-457nt\") pod \"model-serving-api-54d89bd5fc-szsqv\" (UID: \"7e3b4a90-43a1-452c-b74a-1e77fc21a067\") " pod="opendatahub/model-serving-api-54d89bd5fc-szsqv"
Jul 18 01:53:51.700491 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.700488    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.700654 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.700522    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e3b4a90-43a1-452c-b74a-1e77fc21a067-tls-certs\") pod \"model-serving-api-54d89bd5fc-szsqv\" (UID: \"7e3b4a90-43a1-452c-b74a-1e77fc21a067\") " pod="opendatahub/model-serving-api-54d89bd5fc-szsqv"
Jul 18 01:53:51.703033 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.703014    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 18 01:53:51.703157 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.703036    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ctngl\""
Jul 18 01:53:51.703157 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.703114    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e3b4a90-43a1-452c-b74a-1e77fc21a067-tls-certs\") pod \"model-serving-api-54d89bd5fc-szsqv\" (UID: \"7e3b4a90-43a1-452c-b74a-1e77fc21a067\") " pod="opendatahub/model-serving-api-54d89bd5fc-szsqv"
Jul 18 01:53:51.703157 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.703121    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 18 01:53:51.703342 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.703329    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 18 01:53:51.703583 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.703567    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 18 01:53:51.708968 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.708946    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-457nt\" (UniqueName: \"kubernetes.io/projected/7e3b4a90-43a1-452c-b74a-1e77fc21a067-kube-api-access-457nt\") pod \"model-serving-api-54d89bd5fc-szsqv\" (UID: \"7e3b4a90-43a1-452c-b74a-1e77fc21a067\") " pod="opendatahub/model-serving-api-54d89bd5fc-szsqv"
Jul 18 01:53:51.720668 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.720635    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"]
Jul 18 01:53:51.725323 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.725292    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:51.798041 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.798006    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-szsqv"
Jul 18 01:53:51.801344 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.800878    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.801344 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.800916    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-token\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.801344 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.800946    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbcrz\" (UniqueName: \"kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-kube-api-access-jbcrz\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.801344 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.800986    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.801344 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.801046    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-cacerts\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.801344 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.801092    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.801344 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.801158    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d8bf70-1e5b-440d-8f49-6f30eec69859-local-certs\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.827534 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.827491    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:51.902640 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.902039    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-token\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.902640 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.902085    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbcrz\" (UniqueName: \"kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-kube-api-access-jbcrz\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.902640 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.902144    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.902640 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.902168    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-cacerts\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.902640 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.902199    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.902640 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.902248    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d8bf70-1e5b-440d-8f49-6f30eec69859-local-certs\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.902640 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.902305    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.903127 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.903027    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.906116 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.906070    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.908019 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.907966    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d8bf70-1e5b-440d-8f49-6f30eec69859-local-certs\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.908643 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.908531    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-cacerts\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.910338 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.910297    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.911308 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.911202    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-token\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.911500 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.911460    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbcrz\" (UniqueName: \"kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-kube-api-access-jbcrz\") pod \"istiod-openshift-gateway-7574459656-s2mrf\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:51.960047 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:51.959998    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-szsqv"]
Jul 18 01:53:51.964410 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:53:51.964374    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e3b4a90_43a1_452c_b74a_1e77fc21a067.slice/crio-09d04c2a1812a53cc982d0241aed70e6ec84085370b57c55185e0ad856ffbd35 WatchSource:0}: Error finding container 09d04c2a1812a53cc982d0241aed70e6ec84085370b57c55185e0ad856ffbd35: Status 404 returned error can't find the container with id 09d04c2a1812a53cc982d0241aed70e6ec84085370b57c55185e0ad856ffbd35
Jul 18 01:53:52.031765 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:52.031725    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:52.223746 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:52.223715    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"]
Jul 18 01:53:52.226395 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:53:52.226365    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d8bf70_1e5b_440d_8f49_6f30eec69859.slice/crio-72229c99cb4fbe8a7414eac4a4b49a120c0ba7a7f765675fe66dbc755ccc1566 WatchSource:0}: Error finding container 72229c99cb4fbe8a7414eac4a4b49a120c0ba7a7f765675fe66dbc755ccc1566: Status 404 returned error can't find the container with id 72229c99cb4fbe8a7414eac4a4b49a120c0ba7a7f765675fe66dbc755ccc1566
Jul 18 01:53:52.699638 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:52.699595    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:52.712670 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:52.712631    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-szsqv" event={"ID":"7e3b4a90-43a1-452c-b74a-1e77fc21a067","Type":"ContainerStarted","Data":"09d04c2a1812a53cc982d0241aed70e6ec84085370b57c55185e0ad856ffbd35"}
Jul 18 01:53:52.718026 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:52.717952    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf" event={"ID":"d1d8bf70-1e5b-440d-8f49-6f30eec69859","Type":"ContainerStarted","Data":"72229c99cb4fbe8a7414eac4a4b49a120c0ba7a7f765675fe66dbc755ccc1566"}
Jul 18 01:53:56.899844 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:56.899811    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 18 01:53:56.900221 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:56.899873    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 18 01:53:57.749417 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.749381    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8" event={"ID":"e37d7d55-2e3b-4cd2-8c2a-fe5078acd475","Type":"ContainerStarted","Data":"b71e917725e9c1f28e2bde2764c9a3f095759a5e40e656c13d3698c716ff0910"}
Jul 18 01:53:57.749650 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.749516    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8"
Jul 18 01:53:57.751100 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.751074    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf" event={"ID":"d1d8bf70-1e5b-440d-8f49-6f30eec69859","Type":"ContainerStarted","Data":"0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f"}
Jul 18 01:53:57.751225 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.751207    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:57.752638 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.752613    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr" event={"ID":"0bfa6dcf-1126-4847-a901-92d8b90d866f","Type":"ContainerStarted","Data":"a5dddf2645d3225f95bb0bb6c05186cf14dd4881fda0b12bae7f46ec9ce0a3a6"}
Jul 18 01:53:57.753280 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.753251    2552 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-s2mrf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 18 01:53:57.753377 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.753311    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf" podUID="d1d8bf70-1e5b-440d-8f49-6f30eec69859" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 01:53:57.754217 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.754183    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-szsqv" event={"ID":"7e3b4a90-43a1-452c-b74a-1e77fc21a067","Type":"ContainerStarted","Data":"2efc67ee7b5f8c869c67c41e4f438af71317adb46cf7f2e1602a18f31c5bfc3e"}
Jul 18 01:53:57.754344 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.754329    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-szsqv"
Jul 18 01:53:57.769493 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.769447    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8" podStartSLOduration=2.480760015 podStartE2EDuration="7.769412497s" totalImagesPullingTime="5.288652482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:57.767901524 +0000 UTC m=+343.207398456" watchObservedRunningTime="2026-07-18 01:53:57.769412497 +0000 UTC m=+343.208909429"
Jul 18 01:53:57.793221 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.793160    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-szsqv" podStartSLOduration=1.8628187999999999 podStartE2EDuration="6.793140662s" totalImagesPullingTime="4.930321862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:57.790417189 +0000 UTC m=+343.229914122" watchObservedRunningTime="2026-07-18 01:53:57.793140662 +0000 UTC m=+343.232637594"
Jul 18 01:53:57.812488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.812406    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf" podStartSLOduration=2.142052413 podStartE2EDuration="6.812386409s" totalImagesPullingTime="4.670333996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:57.81013167 +0000 UTC m=+343.249628599" watchObservedRunningTime="2026-07-18 01:53:57.812386409 +0000 UTC m=+343.251883341"
Jul 18 01:53:57.831822 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.831768    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58f5976ff9-lftlr" podStartSLOduration=1.7984449599999999 podStartE2EDuration="7.831748071s" totalImagesPullingTime="6.033303111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:57.829194986 +0000 UTC m=+343.268691918" watchObservedRunningTime="2026-07-18 01:53:57.831748071 +0000 UTC m=+343.271245002"
Jul 18 01:53:57.919695 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:57.919630    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:58.023388 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:58.023308    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:58.450348 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:58.450297    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:58.760695 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:58.760616    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:53:58.775601 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:58.775566    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:53:58.969504 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:53:58.969472    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:03.100380 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.100336    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:03.169799 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.169768    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"]
Jul 18 01:54:03.176408 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.176386    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"
Jul 18 01:54:03.179593 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.179576    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ccrj5\""
Jul 18 01:54:03.179781 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.179767    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 18 01:54:03.191330 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.191303    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"]
Jul 18 01:54:03.203250 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.203216    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/378c1a2d-982f-46ce-a506-88f8b567ccbd-cert\") pod \"kserve-controller-manager-d6f4b65b8-v97ql\" (UID: \"378c1a2d-982f-46ce-a506-88f8b567ccbd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"
Jul 18 01:54:03.203373 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.203274    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b68px\" (UniqueName: \"kubernetes.io/projected/378c1a2d-982f-46ce-a506-88f8b567ccbd-kube-api-access-b68px\") pod \"kserve-controller-manager-d6f4b65b8-v97ql\" (UID: \"378c1a2d-982f-46ce-a506-88f8b567ccbd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"
Jul 18 01:54:03.204352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.204335    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"]
Jul 18 01:54:03.204449 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.204437    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"
Jul 18 01:54:03.211750 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.211710    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:03.219844 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.219824    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 18 01:54:03.219967 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.219827    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-7q7wx\""
Jul 18 01:54:03.219967 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.219827    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"]
Jul 18 01:54:03.304479 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.304369    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/378c1a2d-982f-46ce-a506-88f8b567ccbd-cert\") pod \"kserve-controller-manager-d6f4b65b8-v97ql\" (UID: \"378c1a2d-982f-46ce-a506-88f8b567ccbd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"
Jul 18 01:54:03.304645 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:54:03.304492    2552 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 18 01:54:03.304645 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.304518    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b68px\" (UniqueName: \"kubernetes.io/projected/378c1a2d-982f-46ce-a506-88f8b567ccbd-kube-api-access-b68px\") pod \"kserve-controller-manager-d6f4b65b8-v97ql\" (UID: \"378c1a2d-982f-46ce-a506-88f8b567ccbd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"
Jul 18 01:54:03.304645 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:54:03.304567    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/378c1a2d-982f-46ce-a506-88f8b567ccbd-cert podName:378c1a2d-982f-46ce-a506-88f8b567ccbd nodeName:}" failed. No retries permitted until 2026-07-18 01:54:03.804546376 +0000 UTC m=+349.244043297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/378c1a2d-982f-46ce-a506-88f8b567ccbd-cert") pod "kserve-controller-manager-d6f4b65b8-v97ql" (UID: "378c1a2d-982f-46ce-a506-88f8b567ccbd") : secret "kserve-webhook-server-cert" not found
Jul 18 01:54:03.304757 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.304654    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr68k\" (UniqueName: \"kubernetes.io/projected/7d8b250e-4567-461b-abca-397c2ad014b4-kube-api-access-vr68k\") pod \"llmisvc-controller-manager-777c6bd7d5-spg4m\" (UID: \"7d8b250e-4567-461b-abca-397c2ad014b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"
Jul 18 01:54:03.304757 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.304682    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d8b250e-4567-461b-abca-397c2ad014b4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-spg4m\" (UID: \"7d8b250e-4567-461b-abca-397c2ad014b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"
Jul 18 01:54:03.321352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.321316    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b68px\" (UniqueName: \"kubernetes.io/projected/378c1a2d-982f-46ce-a506-88f8b567ccbd-kube-api-access-b68px\") pod \"kserve-controller-manager-d6f4b65b8-v97ql\" (UID: \"378c1a2d-982f-46ce-a506-88f8b567ccbd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"
Jul 18 01:54:03.405071 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.405041    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr68k\" (UniqueName: \"kubernetes.io/projected/7d8b250e-4567-461b-abca-397c2ad014b4-kube-api-access-vr68k\") pod \"llmisvc-controller-manager-777c6bd7d5-spg4m\" (UID: \"7d8b250e-4567-461b-abca-397c2ad014b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"
Jul 18 01:54:03.405257 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.405077    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d8b250e-4567-461b-abca-397c2ad014b4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-spg4m\" (UID: \"7d8b250e-4567-461b-abca-397c2ad014b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"
Jul 18 01:54:03.405257 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:54:03.405237    2552 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 18 01:54:03.405341 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:54:03.405301    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8b250e-4567-461b-abca-397c2ad014b4-cert podName:7d8b250e-4567-461b-abca-397c2ad014b4 nodeName:}" failed. No retries permitted until 2026-07-18 01:54:03.905282824 +0000 UTC m=+349.344779734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7d8b250e-4567-461b-abca-397c2ad014b4-cert") pod "llmisvc-controller-manager-777c6bd7d5-spg4m" (UID: "7d8b250e-4567-461b-abca-397c2ad014b4") : secret "llmisvc-webhook-server-cert" not found
Jul 18 01:54:03.428505 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.428474    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr68k\" (UniqueName: \"kubernetes.io/projected/7d8b250e-4567-461b-abca-397c2ad014b4-kube-api-access-vr68k\") pod \"llmisvc-controller-manager-777c6bd7d5-spg4m\" (UID: \"7d8b250e-4567-461b-abca-397c2ad014b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"
Jul 18 01:54:03.448791 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.448759    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:03.543310 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.543280    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:03.807334 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.807244    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/378c1a2d-982f-46ce-a506-88f8b567ccbd-cert\") pod \"kserve-controller-manager-d6f4b65b8-v97ql\" (UID: \"378c1a2d-982f-46ce-a506-88f8b567ccbd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"
Jul 18 01:54:03.809890 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.809861    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/378c1a2d-982f-46ce-a506-88f8b567ccbd-cert\") pod \"kserve-controller-manager-d6f4b65b8-v97ql\" (UID: \"378c1a2d-982f-46ce-a506-88f8b567ccbd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"
Jul 18 01:54:03.908330 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.908291    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d8b250e-4567-461b-abca-397c2ad014b4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-spg4m\" (UID: \"7d8b250e-4567-461b-abca-397c2ad014b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"
Jul 18 01:54:03.910915 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:03.910883    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d8b250e-4567-461b-abca-397c2ad014b4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-spg4m\" (UID: \"7d8b250e-4567-461b-abca-397c2ad014b4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"
Jul 18 01:54:04.086061 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:04.085978    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"
Jul 18 01:54:04.114786 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:04.114755    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"
Jul 18 01:54:04.273291 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:04.273270    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"]
Jul 18 01:54:04.274952 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:54:04.274927    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod378c1a2d_982f_46ce_a506_88f8b567ccbd.slice/crio-716ccbd51b9f685b212e4e09fc0047911187526411598da9e3dd24c43eb9f23c WatchSource:0}: Error finding container 716ccbd51b9f685b212e4e09fc0047911187526411598da9e3dd24c43eb9f23c: Status 404 returned error can't find the container with id 716ccbd51b9f685b212e4e09fc0047911187526411598da9e3dd24c43eb9f23c
Jul 18 01:54:04.301262 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:04.301238    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"]
Jul 18 01:54:04.302789 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:54:04.302762    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7d8b250e_4567_461b_abca_397c2ad014b4.slice/crio-b8917261bb94eca98ca094c7fbabc36d680af293a6fa629ae26e68b59870b12e WatchSource:0}: Error finding container b8917261bb94eca98ca094c7fbabc36d680af293a6fa629ae26e68b59870b12e: Status 404 returned error can't find the container with id b8917261bb94eca98ca094c7fbabc36d680af293a6fa629ae26e68b59870b12e
Jul 18 01:54:04.465886 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:04.465852    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:04.562641 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:04.562606    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:04.641502 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:04.641471    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:04.788363 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:04.788252    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m" event={"ID":"7d8b250e-4567-461b-abca-397c2ad014b4","Type":"ContainerStarted","Data":"b8917261bb94eca98ca094c7fbabc36d680af293a6fa629ae26e68b59870b12e"}
Jul 18 01:54:04.789566 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:04.789541    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql" event={"ID":"378c1a2d-982f-46ce-a506-88f8b567ccbd","Type":"ContainerStarted","Data":"716ccbd51b9f685b212e4e09fc0047911187526411598da9e3dd24c43eb9f23c"}
Jul 18 01:54:04.789925 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:04.789909    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"]
Jul 18 01:54:04.790089 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:04.790072    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf" podUID="d1d8bf70-1e5b-440d-8f49-6f30eec69859" containerName="discovery" containerID="cri-o://0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f" gracePeriod=30
Jul 18 01:54:05.090843 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.090815    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:54:05.117793 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.117766    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-cacerts\") pod \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") "
Jul 18 01:54:05.118352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.117799    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-dns-cert\") pod \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") "
Jul 18 01:54:05.118352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.117843    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-ca-configmap\") pod \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") "
Jul 18 01:54:05.118352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.117869    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-token\") pod \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") "
Jul 18 01:54:05.118352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.117919    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-kubeconfig\") pod \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") "
Jul 18 01:54:05.118352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.117962    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbcrz\" (UniqueName: \"kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-kube-api-access-jbcrz\") pod \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") "
Jul 18 01:54:05.118352 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.118006    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d8bf70-1e5b-440d-8f49-6f30eec69859-local-certs\") pod \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\" (UID: \"d1d8bf70-1e5b-440d-8f49-6f30eec69859\") "
Jul 18 01:54:05.118899 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.118870    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "d1d8bf70-1e5b-440d-8f49-6f30eec69859" (UID: "d1d8bf70-1e5b-440d-8f49-6f30eec69859"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:54:05.121536 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.121509    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "d1d8bf70-1e5b-440d-8f49-6f30eec69859" (UID: "d1d8bf70-1e5b-440d-8f49-6f30eec69859"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:54:05.121655 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.121533    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "d1d8bf70-1e5b-440d-8f49-6f30eec69859" (UID: "d1d8bf70-1e5b-440d-8f49-6f30eec69859"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:54:05.122174 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.122147    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-cacerts" (OuterVolumeSpecName: "cacerts") pod "d1d8bf70-1e5b-440d-8f49-6f30eec69859" (UID: "d1d8bf70-1e5b-440d-8f49-6f30eec69859"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:54:05.122655 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.122627    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-kube-api-access-jbcrz" (OuterVolumeSpecName: "kube-api-access-jbcrz") pod "d1d8bf70-1e5b-440d-8f49-6f30eec69859" (UID: "d1d8bf70-1e5b-440d-8f49-6f30eec69859"). InnerVolumeSpecName "kube-api-access-jbcrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:54:05.123240 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.123198    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d8bf70-1e5b-440d-8f49-6f30eec69859-local-certs" (OuterVolumeSpecName: "local-certs") pod "d1d8bf70-1e5b-440d-8f49-6f30eec69859" (UID: "d1d8bf70-1e5b-440d-8f49-6f30eec69859"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:54:05.123873 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.123848    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-token" (OuterVolumeSpecName: "istio-token") pod "d1d8bf70-1e5b-440d-8f49-6f30eec69859" (UID: "d1d8bf70-1e5b-440d-8f49-6f30eec69859"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:54:05.218786 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.218752    2552 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-cacerts\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:54:05.218786 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.218782    2552 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-dns-cert\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:54:05.218786 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.218793    2552 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-csr-ca-configmap\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:54:05.219095 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.218803    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-token\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:54:05.219095 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.218812    2552 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1d8bf70-1e5b-440d-8f49-6f30eec69859-istio-kubeconfig\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:54:05.219095 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.218820    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbcrz\" (UniqueName: \"kubernetes.io/projected/d1d8bf70-1e5b-440d-8f49-6f30eec69859-kube-api-access-jbcrz\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:54:05.219095 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.218834    2552 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d8bf70-1e5b-440d-8f49-6f30eec69859-local-certs\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:54:05.797936 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.797838    2552 generic.go:356] "Generic (PLEG): container finished" podID="d1d8bf70-1e5b-440d-8f49-6f30eec69859" containerID="0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f" exitCode=0
Jul 18 01:54:05.797936 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.797916    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"
Jul 18 01:54:05.797936 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.797921    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf" event={"ID":"d1d8bf70-1e5b-440d-8f49-6f30eec69859","Type":"ContainerDied","Data":"0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f"}
Jul 18 01:54:05.798203 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.798020    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf" event={"ID":"d1d8bf70-1e5b-440d-8f49-6f30eec69859","Type":"ContainerDied","Data":"72229c99cb4fbe8a7414eac4a4b49a120c0ba7a7f765675fe66dbc755ccc1566"}
Jul 18 01:54:05.798203 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.798051    2552 scope.go:117] "RemoveContainer" containerID="0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f"
Jul 18 01:54:05.810231 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.810207    2552 scope.go:117] "RemoveContainer" containerID="0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f"
Jul 18 01:54:05.810645 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:54:05.810617    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f\": container with ID starting with 0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f not found: ID does not exist" containerID="0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f"
Jul 18 01:54:05.810742 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.810652    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f"} err="failed to get container status \"0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f\": rpc error: code = NotFound desc = could not find container \"0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f\": container with ID starting with 0c9c7283a533c045248230f717f1a4d6361ac5d977859e156ae02ebb967f192f not found: ID does not exist"
Jul 18 01:54:05.825016 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.824987    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"]
Jul 18 01:54:05.829591 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:05.829563    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-s2mrf"]
Jul 18 01:54:06.212367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:06.212301    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:07.078181 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:07.078145    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1d8bf70-1e5b-440d-8f49-6f30eec69859" path="/var/lib/kubelet/pods/d1d8bf70-1e5b-440d-8f49-6f30eec69859/volumes"
Jul 18 01:54:07.812055 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:07.811977    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m" event={"ID":"7d8b250e-4567-461b-abca-397c2ad014b4","Type":"ContainerStarted","Data":"d69f4bc77c5bcb47dbf9061f068778730cb1ae44bafbff954e0c1a6820493751"}
Jul 18 01:54:07.812618 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:07.812104    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"
Jul 18 01:54:07.813735 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:07.813701    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql" event={"ID":"378c1a2d-982f-46ce-a506-88f8b567ccbd","Type":"ContainerStarted","Data":"3a5845178ef6a24cbcea1b926841265aea59783ff411ef94f47bae2a5b0e9782"}
Jul 18 01:54:07.813866 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:07.813790    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"
Jul 18 01:54:07.829390 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:07.829342    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m" podStartSLOduration=1.738005475 podStartE2EDuration="4.829328823s" totalImagesPullingTime="3.091323348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:07.827565505 +0000 UTC m=+353.267062457" watchObservedRunningTime="2026-07-18 01:54:07.829328823 +0000 UTC m=+353.268825773"
Jul 18 01:54:07.844937 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:07.844894    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql" podStartSLOduration=1.724329837 podStartE2EDuration="4.844881058s" totalImagesPullingTime="3.120551221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:07.843219951 +0000 UTC m=+353.282716885" watchObservedRunningTime="2026-07-18 01:54:07.844881058 +0000 UTC m=+353.284377989"
Jul 18 01:54:08.762838 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:08.762811    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-qcsz8"
Jul 18 01:54:08.764689 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:08.764668    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-szsqv"
Jul 18 01:54:38.822482 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:38.822453    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-spg4m"
Jul 18 01:54:38.825491 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:38.825469    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v97ql"
Jul 18 01:54:51.156819 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.156779    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:51.256922 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.256891    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:51.301184 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.301148    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kksth"]
Jul 18 01:54:51.301509 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.301497    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1d8bf70-1e5b-440d-8f49-6f30eec69859" containerName="discovery"
Jul 18 01:54:51.301559 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.301513    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1d8bf70-1e5b-440d-8f49-6f30eec69859" containerName="discovery"
Jul 18 01:54:51.301620 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.301609    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1d8bf70-1e5b-440d-8f49-6f30eec69859" containerName="discovery"
Jul 18 01:54:51.306557 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.306533    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kksth"
Jul 18 01:54:51.308747 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.308723    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-ln9gd\""
Jul 18 01:54:51.309532 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.309510    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 18 01:54:51.309715 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.309537    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 18 01:54:51.317874 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.317853    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kksth"]
Jul 18 01:54:51.353975 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.353944    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:54:51.408503 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.408397    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvcd7\" (UniqueName: \"kubernetes.io/projected/440abf8e-7151-4835-8036-97049a6d9b19-kube-api-access-mvcd7\") pod \"kuadrant-operator-catalog-kksth\" (UID: \"440abf8e-7151-4835-8036-97049a6d9b19\") " pod="kuadrant-system/kuadrant-operator-catalog-kksth"
Jul 18 01:54:51.509269 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.509240    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvcd7\" (UniqueName: \"kubernetes.io/projected/440abf8e-7151-4835-8036-97049a6d9b19-kube-api-access-mvcd7\") pod \"kuadrant-operator-catalog-kksth\" (UID: \"440abf8e-7151-4835-8036-97049a6d9b19\") " pod="kuadrant-system/kuadrant-operator-catalog-kksth"
Jul 18 01:54:51.517725 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.517693    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvcd7\" (UniqueName: \"kubernetes.io/projected/440abf8e-7151-4835-8036-97049a6d9b19-kube-api-access-mvcd7\") pod \"kuadrant-operator-catalog-kksth\" (UID: \"440abf8e-7151-4835-8036-97049a6d9b19\") " pod="kuadrant-system/kuadrant-operator-catalog-kksth"
Jul 18 01:54:51.617605 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.617568    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kksth"
Jul 18 01:54:51.658978 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.658893    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kksth"]
Jul 18 01:54:51.767248 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.765640    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kksth"]
Jul 18 01:54:51.769387 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:54:51.769345    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440abf8e_7151_4835_8036_97049a6d9b19.slice/crio-6dd1f05235c2345e01d565a858487e62506a5dcee20a3dd5696be3c2483a8d8d WatchSource:0}: Error finding container 6dd1f05235c2345e01d565a858487e62506a5dcee20a3dd5696be3c2483a8d8d: Status 404 returned error can't find the container with id 6dd1f05235c2345e01d565a858487e62506a5dcee20a3dd5696be3c2483a8d8d
Jul 18 01:54:51.886016 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.885984    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-slzb6"]
Jul 18 01:54:51.888792 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.888776    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-slzb6"
Jul 18 01:54:51.898751 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:51.898726    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-slzb6"]
Jul 18 01:54:52.009815 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:52.009731    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kksth" event={"ID":"440abf8e-7151-4835-8036-97049a6d9b19","Type":"ContainerStarted","Data":"6dd1f05235c2345e01d565a858487e62506a5dcee20a3dd5696be3c2483a8d8d"}
Jul 18 01:54:52.013098 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:52.013064    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c295k\" (UniqueName: \"kubernetes.io/projected/de2671e7-b13e-4801-af5b-a66419d98590-kube-api-access-c295k\") pod \"kuadrant-operator-catalog-slzb6\" (UID: \"de2671e7-b13e-4801-af5b-a66419d98590\") " pod="kuadrant-system/kuadrant-operator-catalog-slzb6"
Jul 18 01:54:52.113966 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:52.113934    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c295k\" (UniqueName: \"kubernetes.io/projected/de2671e7-b13e-4801-af5b-a66419d98590-kube-api-access-c295k\") pod \"kuadrant-operator-catalog-slzb6\" (UID: \"de2671e7-b13e-4801-af5b-a66419d98590\") " pod="kuadrant-system/kuadrant-operator-catalog-slzb6"
Jul 18 01:54:52.122942 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:52.122903    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c295k\" (UniqueName: \"kubernetes.io/projected/de2671e7-b13e-4801-af5b-a66419d98590-kube-api-access-c295k\") pod \"kuadrant-operator-catalog-slzb6\" (UID: \"de2671e7-b13e-4801-af5b-a66419d98590\") " pod="kuadrant-system/kuadrant-operator-catalog-slzb6"
Jul 18 01:54:52.198373 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:52.198338    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-slzb6"
Jul 18 01:54:52.361551 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:52.361467    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-slzb6"]
Jul 18 01:54:52.363775 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:54:52.363748    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2671e7_b13e_4801_af5b_a66419d98590.slice/crio-3de5c19b55b8261d5d45d574ed88718debbaff2bd29c7722b9e7bea6123acfa2 WatchSource:0}: Error finding container 3de5c19b55b8261d5d45d574ed88718debbaff2bd29c7722b9e7bea6123acfa2: Status 404 returned error can't find the container with id 3de5c19b55b8261d5d45d574ed88718debbaff2bd29c7722b9e7bea6123acfa2
Jul 18 01:54:53.017379 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:53.017344    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-slzb6" event={"ID":"de2671e7-b13e-4801-af5b-a66419d98590","Type":"ContainerStarted","Data":"3de5c19b55b8261d5d45d574ed88718debbaff2bd29c7722b9e7bea6123acfa2"}
Jul 18 01:54:55.027752 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:55.027716    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-slzb6" event={"ID":"de2671e7-b13e-4801-af5b-a66419d98590","Type":"ContainerStarted","Data":"b5d4af8747f7f96b252ffa017f667bd74af9d0701fee84a593e04791a3375b60"}
Jul 18 01:54:55.029152 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:55.029126    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kksth" event={"ID":"440abf8e-7151-4835-8036-97049a6d9b19","Type":"ContainerStarted","Data":"245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7"}
Jul 18 01:54:55.029260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:55.029207    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-kksth" podUID="440abf8e-7151-4835-8036-97049a6d9b19" containerName="registry-server" containerID="cri-o://245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7" gracePeriod=2
Jul 18 01:54:55.044383 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:55.044112    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-slzb6" podStartSLOduration=2.382392937 podStartE2EDuration="4.04409311s" totalImagesPullingTime="1.661700173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:55.042776062 +0000 UTC m=+400.482272995" watchObservedRunningTime="2026-07-18 01:54:55.04409311 +0000 UTC m=+400.483590043"
Jul 18 01:54:55.063733 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:55.063680    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kksth" podStartSLOduration=1.808240482 podStartE2EDuration="4.063665337s" totalImagesPullingTime="2.255424855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:55.062287025 +0000 UTC m=+400.501783958" watchObservedRunningTime="2026-07-18 01:54:55.063665337 +0000 UTC m=+400.503162269"
Jul 18 01:54:55.279665 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:55.279607    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kksth"
Jul 18 01:54:55.443575 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:55.443534    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvcd7\" (UniqueName: \"kubernetes.io/projected/440abf8e-7151-4835-8036-97049a6d9b19-kube-api-access-mvcd7\") pod \"440abf8e-7151-4835-8036-97049a6d9b19\" (UID: \"440abf8e-7151-4835-8036-97049a6d9b19\") "
Jul 18 01:54:55.445890 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:55.445861    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/440abf8e-7151-4835-8036-97049a6d9b19-kube-api-access-mvcd7" (OuterVolumeSpecName: "kube-api-access-mvcd7") pod "440abf8e-7151-4835-8036-97049a6d9b19" (UID: "440abf8e-7151-4835-8036-97049a6d9b19"). InnerVolumeSpecName "kube-api-access-mvcd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:54:55.544781 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:55.544700    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvcd7\" (UniqueName: \"kubernetes.io/projected/440abf8e-7151-4835-8036-97049a6d9b19-kube-api-access-mvcd7\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:54:56.035073 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:56.035024    2552 generic.go:356] "Generic (PLEG): container finished" podID="440abf8e-7151-4835-8036-97049a6d9b19" containerID="245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7" exitCode=0
Jul 18 01:54:56.035073 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:56.035080    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kksth"
Jul 18 01:54:56.035533 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:56.035112    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kksth" event={"ID":"440abf8e-7151-4835-8036-97049a6d9b19","Type":"ContainerDied","Data":"245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7"}
Jul 18 01:54:56.035533 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:56.035162    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kksth" event={"ID":"440abf8e-7151-4835-8036-97049a6d9b19","Type":"ContainerDied","Data":"6dd1f05235c2345e01d565a858487e62506a5dcee20a3dd5696be3c2483a8d8d"}
Jul 18 01:54:56.035533 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:56.035181    2552 scope.go:117] "RemoveContainer" containerID="245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7"
Jul 18 01:54:56.044225 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:56.044208    2552 scope.go:117] "RemoveContainer" containerID="245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7"
Jul 18 01:54:56.044518 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:54:56.044501    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7\": container with ID starting with 245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7 not found: ID does not exist" containerID="245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7"
Jul 18 01:54:56.044584 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:56.044530    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7"} err="failed to get container status \"245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7\": rpc error: code = NotFound desc = could not find container \"245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7\": container with ID starting with 245e4f104821eb2ded51243cf11418bd25e007565309de8bdead2de1c227bbd7 not found: ID does not exist"
Jul 18 01:54:56.054830 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:56.054807    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kksth"]
Jul 18 01:54:56.056953 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:56.056931    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kksth"]
Jul 18 01:54:57.075924 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:54:57.075887    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="440abf8e-7151-4835-8036-97049a6d9b19" path="/var/lib/kubelet/pods/440abf8e-7151-4835-8036-97049a6d9b19/volumes"
Jul 18 01:55:02.199465 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:02.199408    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-slzb6"
Jul 18 01:55:02.199950 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:02.199489    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-slzb6"
Jul 18 01:55:02.222274 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:02.222245    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-slzb6"
Jul 18 01:55:03.091410 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:03.091384    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-slzb6"
Jul 18 01:55:06.695030 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.694997    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"]
Jul 18 01:55:06.695416 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.695353    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="440abf8e-7151-4835-8036-97049a6d9b19" containerName="registry-server"
Jul 18 01:55:06.695416 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.695368    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="440abf8e-7151-4835-8036-97049a6d9b19" containerName="registry-server"
Jul 18 01:55:06.695509 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.695446    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="440abf8e-7151-4835-8036-97049a6d9b19" containerName="registry-server"
Jul 18 01:55:06.698328 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.698312    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:06.700623 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.700603    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qp2sp\""
Jul 18 01:55:06.712165 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.712141    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"]
Jul 18 01:55:06.733983 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.733952    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:06.733983 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.733989    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:06.734182 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.734069    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbk7p\" (UniqueName: \"kubernetes.io/projected/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-kube-api-access-cbk7p\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:06.835369 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.835323    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbk7p\" (UniqueName: \"kubernetes.io/projected/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-kube-api-access-cbk7p\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:06.835369 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.835383    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:06.835607 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.835417    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:06.835885 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.835859    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:06.835922 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.835868    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:06.844885 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:06.844850    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbk7p\" (UniqueName: \"kubernetes.io/projected/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-kube-api-access-cbk7p\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:07.006845 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.006757    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:07.145348 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.145318    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"]
Jul 18 01:55:07.148911 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:07.148885    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c33b59c_9f76_4033_88ca_9a2eab6b09ee.slice/crio-e6480352a2abf4200d33e46ce99dad8c8539da33cf832d1b7cefbdc405e034d6 WatchSource:0}: Error finding container e6480352a2abf4200d33e46ce99dad8c8539da33cf832d1b7cefbdc405e034d6: Status 404 returned error can't find the container with id e6480352a2abf4200d33e46ce99dad8c8539da33cf832d1b7cefbdc405e034d6
Jul 18 01:55:07.308059 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.307989    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"]
Jul 18 01:55:07.311248 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.311232    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:07.318789 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.318765    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"]
Jul 18 01:55:07.339435 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.339385    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:07.339578 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.339451    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:07.339578 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.339510    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf65d\" (UniqueName: \"kubernetes.io/projected/9a0680e2-b156-4327-ae19-2ef7c9c508dd-kube-api-access-xf65d\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:07.440175 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.440136    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:07.440327 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.440183    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf65d\" (UniqueName: \"kubernetes.io/projected/9a0680e2-b156-4327-ae19-2ef7c9c508dd-kube-api-access-xf65d\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:07.440327 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.440228    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:07.440555 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.440536    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:07.440599 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.440564    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:07.448461 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.448412    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf65d\" (UniqueName: \"kubernetes.io/projected/9a0680e2-b156-4327-ae19-2ef7c9c508dd-kube-api-access-xf65d\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:07.621386 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.621305    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:07.694677 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.694648    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"]
Jul 18 01:55:07.699125 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.699102    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:07.708644 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.708332    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"]
Jul 18 01:55:07.742697 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.742667    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:07.742849 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.742710    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:07.742849 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.742776    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6zbm\" (UniqueName: \"kubernetes.io/projected/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-kube-api-access-s6zbm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:07.766735 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.766654    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"]
Jul 18 01:55:07.768922 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:07.768897    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a0680e2_b156_4327_ae19_2ef7c9c508dd.slice/crio-76367ce6e9000061681340ed56ac8db0e3c2bcf664537b90382fc17f92f7770a WatchSource:0}: Error finding container 76367ce6e9000061681340ed56ac8db0e3c2bcf664537b90382fc17f92f7770a: Status 404 returned error can't find the container with id 76367ce6e9000061681340ed56ac8db0e3c2bcf664537b90382fc17f92f7770a
Jul 18 01:55:07.843235 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.843201    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:07.843411 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.843247    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:07.843411 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.843278    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6zbm\" (UniqueName: \"kubernetes.io/projected/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-kube-api-access-s6zbm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:07.843667 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.843646    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:07.843734 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.843713    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:07.852247 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:07.852219    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6zbm\" (UniqueName: \"kubernetes.io/projected/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-kube-api-access-s6zbm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:08.010656 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.010622    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:08.093854 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.093817    2552 generic.go:356] "Generic (PLEG): container finished" podID="9a0680e2-b156-4327-ae19-2ef7c9c508dd" containerID="3c6785c689878e49ae3a9a49966c73126124bf950090ac8810665e4c9a4fca92" exitCode=0
Jul 18 01:55:08.094035 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.093901    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz" event={"ID":"9a0680e2-b156-4327-ae19-2ef7c9c508dd","Type":"ContainerDied","Data":"3c6785c689878e49ae3a9a49966c73126124bf950090ac8810665e4c9a4fca92"}
Jul 18 01:55:08.094035 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.093960    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz" event={"ID":"9a0680e2-b156-4327-ae19-2ef7c9c508dd","Type":"ContainerStarted","Data":"76367ce6e9000061681340ed56ac8db0e3c2bcf664537b90382fc17f92f7770a"}
Jul 18 01:55:08.096192 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.096067    2552 generic.go:356] "Generic (PLEG): container finished" podID="0c33b59c-9f76-4033-88ca-9a2eab6b09ee" containerID="85e5f1df280f31f22dc45b4e2598e5f756d23d794bf36e00a6d20a6c48cbe569" exitCode=0
Jul 18 01:55:08.096192 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.096101    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn" event={"ID":"0c33b59c-9f76-4033-88ca-9a2eab6b09ee","Type":"ContainerDied","Data":"85e5f1df280f31f22dc45b4e2598e5f756d23d794bf36e00a6d20a6c48cbe569"}
Jul 18 01:55:08.096192 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.096140    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn" event={"ID":"0c33b59c-9f76-4033-88ca-9a2eab6b09ee","Type":"ContainerStarted","Data":"e6480352a2abf4200d33e46ce99dad8c8539da33cf832d1b7cefbdc405e034d6"}
Jul 18 01:55:08.101386 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.101357    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"]
Jul 18 01:55:08.106284 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.106264    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:08.114287 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.114258    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"]
Jul 18 01:55:08.145450 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.145404    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:08.145586 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.145504    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:08.145586 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.145530    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57lt8\" (UniqueName: \"kubernetes.io/projected/ea3b3415-162b-40bb-84e2-27afe590196b-kube-api-access-57lt8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:08.154518 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.154498    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"]
Jul 18 01:55:08.156865 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:08.156842    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d7ee84_e2dc_4c36_ab7b_95085eeb39d1.slice/crio-c53a6d148f138f49ff1aa3d27758a05725026cb40fc0a5d3a431322dbd85bbdf WatchSource:0}: Error finding container c53a6d148f138f49ff1aa3d27758a05725026cb40fc0a5d3a431322dbd85bbdf: Status 404 returned error can't find the container with id c53a6d148f138f49ff1aa3d27758a05725026cb40fc0a5d3a431322dbd85bbdf
Jul 18 01:55:08.247244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.247196    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:08.247244 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.247241    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57lt8\" (UniqueName: \"kubernetes.io/projected/ea3b3415-162b-40bb-84e2-27afe590196b-kube-api-access-57lt8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:08.247539 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.247313    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:08.247660 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.247629    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:08.247721 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.247677    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:08.256831 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.256804    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57lt8\" (UniqueName: \"kubernetes.io/projected/ea3b3415-162b-40bb-84e2-27afe590196b-kube-api-access-57lt8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:08.419415 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.419383    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:08.555127 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:08.555106    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"]
Jul 18 01:55:08.590141 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:08.590108    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3b3415_162b_40bb_84e2_27afe590196b.slice/crio-b775c9a9099987465e5613c9bf50b539e846e9de84e45003f91d0f46cf47ab51 WatchSource:0}: Error finding container b775c9a9099987465e5613c9bf50b539e846e9de84e45003f91d0f46cf47ab51: Status 404 returned error can't find the container with id b775c9a9099987465e5613c9bf50b539e846e9de84e45003f91d0f46cf47ab51
Jul 18 01:55:09.102207 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:09.102109    2552 generic.go:356] "Generic (PLEG): container finished" podID="a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" containerID="89beccb4234cfb97c25802a8fd0b004e8c2673a11b3c0cf73caadec79ab5d3aa" exitCode=0
Jul 18 01:55:09.102207 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:09.102186    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk" event={"ID":"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1","Type":"ContainerDied","Data":"89beccb4234cfb97c25802a8fd0b004e8c2673a11b3c0cf73caadec79ab5d3aa"}
Jul 18 01:55:09.102734 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:09.102233    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk" event={"ID":"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1","Type":"ContainerStarted","Data":"c53a6d148f138f49ff1aa3d27758a05725026cb40fc0a5d3a431322dbd85bbdf"}
Jul 18 01:55:09.104283 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:09.104249    2552 generic.go:356] "Generic (PLEG): container finished" podID="9a0680e2-b156-4327-ae19-2ef7c9c508dd" containerID="22585c2984f0738959c3c66472949ba6f549ff6a331fe37e4ae8943ca46f260f" exitCode=0
Jul 18 01:55:09.104376 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:09.104306    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz" event={"ID":"9a0680e2-b156-4327-ae19-2ef7c9c508dd","Type":"ContainerDied","Data":"22585c2984f0738959c3c66472949ba6f549ff6a331fe37e4ae8943ca46f260f"}
Jul 18 01:55:09.105894 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:09.105867    2552 generic.go:356] "Generic (PLEG): container finished" podID="ea3b3415-162b-40bb-84e2-27afe590196b" containerID="c06c73af4a0eabbd65f6e63069ba882a2b7b03a2e237e2b45ba431fd22c40230" exitCode=0
Jul 18 01:55:09.105963 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:09.105902    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f" event={"ID":"ea3b3415-162b-40bb-84e2-27afe590196b","Type":"ContainerDied","Data":"c06c73af4a0eabbd65f6e63069ba882a2b7b03a2e237e2b45ba431fd22c40230"}
Jul 18 01:55:09.105963 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:09.105936    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f" event={"ID":"ea3b3415-162b-40bb-84e2-27afe590196b","Type":"ContainerStarted","Data":"b775c9a9099987465e5613c9bf50b539e846e9de84e45003f91d0f46cf47ab51"}
Jul 18 01:55:09.107909 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:09.107891    2552 generic.go:356] "Generic (PLEG): container finished" podID="0c33b59c-9f76-4033-88ca-9a2eab6b09ee" containerID="52293b9e88cadbad111bdd5bcbab0f6bd0a5bb0f5d38a89a4a44d98e3860994b" exitCode=0
Jul 18 01:55:09.108013 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:09.107939    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn" event={"ID":"0c33b59c-9f76-4033-88ca-9a2eab6b09ee","Type":"ContainerDied","Data":"52293b9e88cadbad111bdd5bcbab0f6bd0a5bb0f5d38a89a4a44d98e3860994b"}
Jul 18 01:55:10.115761 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:10.115667    2552 generic.go:356] "Generic (PLEG): container finished" podID="9a0680e2-b156-4327-ae19-2ef7c9c508dd" containerID="fa689a4ff3a38ba310ff42d30096c5ec6e312a8d7f85c09f94d0a2ef50160bf3" exitCode=0
Jul 18 01:55:10.115761 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:10.115745    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz" event={"ID":"9a0680e2-b156-4327-ae19-2ef7c9c508dd","Type":"ContainerDied","Data":"fa689a4ff3a38ba310ff42d30096c5ec6e312a8d7f85c09f94d0a2ef50160bf3"}
Jul 18 01:55:10.117390 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:10.117370    2552 generic.go:356] "Generic (PLEG): container finished" podID="ea3b3415-162b-40bb-84e2-27afe590196b" containerID="b2537c50dac08f39d703e1e8c1a9563ba375c01be247b159063081d5ea3ff5db" exitCode=0
Jul 18 01:55:10.117518 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:10.117466    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f" event={"ID":"ea3b3415-162b-40bb-84e2-27afe590196b","Type":"ContainerDied","Data":"b2537c50dac08f39d703e1e8c1a9563ba375c01be247b159063081d5ea3ff5db"}
Jul 18 01:55:10.119657 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:10.119629    2552 generic.go:356] "Generic (PLEG): container finished" podID="0c33b59c-9f76-4033-88ca-9a2eab6b09ee" containerID="c22d6df26b2209747259ae14c55a2ce13e1da39b8efb2fb6422e16f94f08e9e0" exitCode=0
Jul 18 01:55:10.119740 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:10.119654    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn" event={"ID":"0c33b59c-9f76-4033-88ca-9a2eab6b09ee","Type":"ContainerDied","Data":"c22d6df26b2209747259ae14c55a2ce13e1da39b8efb2fb6422e16f94f08e9e0"}
Jul 18 01:55:10.121398 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:10.121378    2552 generic.go:356] "Generic (PLEG): container finished" podID="a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" containerID="a83895c23703dda2e195a5f7ddffa72ac95d621c0cb44c43cca57df27f6cb296" exitCode=0
Jul 18 01:55:10.121500 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:10.121452    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk" event={"ID":"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1","Type":"ContainerDied","Data":"a83895c23703dda2e195a5f7ddffa72ac95d621c0cb44c43cca57df27f6cb296"}
Jul 18 01:55:11.128221 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.128187    2552 generic.go:356] "Generic (PLEG): container finished" podID="a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" containerID="750b00b4f6d49f83aa90c8c8e9db54448076f70743dbc5178a064822ce2f809d" exitCode=0
Jul 18 01:55:11.128653 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.128269    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk" event={"ID":"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1","Type":"ContainerDied","Data":"750b00b4f6d49f83aa90c8c8e9db54448076f70743dbc5178a064822ce2f809d"}
Jul 18 01:55:11.130302 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.130279    2552 generic.go:356] "Generic (PLEG): container finished" podID="ea3b3415-162b-40bb-84e2-27afe590196b" containerID="6d0f2d91161586f45721358b1e7824880ab64392252c490bea094a13440c93eb" exitCode=0
Jul 18 01:55:11.130435 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.130363    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f" event={"ID":"ea3b3415-162b-40bb-84e2-27afe590196b","Type":"ContainerDied","Data":"6d0f2d91161586f45721358b1e7824880ab64392252c490bea094a13440c93eb"}
Jul 18 01:55:11.264385 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.264358    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:11.292222 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.292197    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:11.374823 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.374797    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbk7p\" (UniqueName: \"kubernetes.io/projected/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-kube-api-access-cbk7p\") pod \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") "
Jul 18 01:55:11.374958 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.374852    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-util\") pod \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") "
Jul 18 01:55:11.374958 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.374888    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-bundle\") pod \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") "
Jul 18 01:55:11.374958 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.374926    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf65d\" (UniqueName: \"kubernetes.io/projected/9a0680e2-b156-4327-ae19-2ef7c9c508dd-kube-api-access-xf65d\") pod \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\" (UID: \"9a0680e2-b156-4327-ae19-2ef7c9c508dd\") "
Jul 18 01:55:11.375192 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.374965    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-util\") pod \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") "
Jul 18 01:55:11.375192 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.374992    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-bundle\") pod \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\" (UID: \"0c33b59c-9f76-4033-88ca-9a2eab6b09ee\") "
Jul 18 01:55:11.375615 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.375567    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-bundle" (OuterVolumeSpecName: "bundle") pod "0c33b59c-9f76-4033-88ca-9a2eab6b09ee" (UID: "0c33b59c-9f76-4033-88ca-9a2eab6b09ee"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:55:11.375736 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.375675    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-bundle" (OuterVolumeSpecName: "bundle") pod "9a0680e2-b156-4327-ae19-2ef7c9c508dd" (UID: "9a0680e2-b156-4327-ae19-2ef7c9c508dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:55:11.377099 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.377072    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a0680e2-b156-4327-ae19-2ef7c9c508dd-kube-api-access-xf65d" (OuterVolumeSpecName: "kube-api-access-xf65d") pod "9a0680e2-b156-4327-ae19-2ef7c9c508dd" (UID: "9a0680e2-b156-4327-ae19-2ef7c9c508dd"). InnerVolumeSpecName "kube-api-access-xf65d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:55:11.377407 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.377375    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-kube-api-access-cbk7p" (OuterVolumeSpecName: "kube-api-access-cbk7p") pod "0c33b59c-9f76-4033-88ca-9a2eab6b09ee" (UID: "0c33b59c-9f76-4033-88ca-9a2eab6b09ee"). InnerVolumeSpecName "kube-api-access-cbk7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:55:11.381162 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.381118    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-util" (OuterVolumeSpecName: "util") pod "0c33b59c-9f76-4033-88ca-9a2eab6b09ee" (UID: "0c33b59c-9f76-4033-88ca-9a2eab6b09ee"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:55:11.381528 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.381509    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-util" (OuterVolumeSpecName: "util") pod "9a0680e2-b156-4327-ae19-2ef7c9c508dd" (UID: "9a0680e2-b156-4327-ae19-2ef7c9c508dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:55:11.476058 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.476023    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-bundle\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:11.476058 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.476053    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf65d\" (UniqueName: \"kubernetes.io/projected/9a0680e2-b156-4327-ae19-2ef7c9c508dd-kube-api-access-xf65d\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:11.476058 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.476062    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-util\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:11.476278 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.476071    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-bundle\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:11.476278 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.476080    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbk7p\" (UniqueName: \"kubernetes.io/projected/0c33b59c-9f76-4033-88ca-9a2eab6b09ee-kube-api-access-cbk7p\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:11.476278 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:11.476088    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a0680e2-b156-4327-ae19-2ef7c9c508dd-util\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:12.137361 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.137333    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn"
Jul 18 01:55:12.137848 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.137369    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn" event={"ID":"0c33b59c-9f76-4033-88ca-9a2eab6b09ee","Type":"ContainerDied","Data":"e6480352a2abf4200d33e46ce99dad8c8539da33cf832d1b7cefbdc405e034d6"}
Jul 18 01:55:12.137848 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.137447    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6480352a2abf4200d33e46ce99dad8c8539da33cf832d1b7cefbdc405e034d6"
Jul 18 01:55:12.139479 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.139453    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz" event={"ID":"9a0680e2-b156-4327-ae19-2ef7c9c508dd","Type":"ContainerDied","Data":"76367ce6e9000061681340ed56ac8db0e3c2bcf664537b90382fc17f92f7770a"}
Jul 18 01:55:12.139479 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.139469    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz"
Jul 18 01:55:12.139628 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.139495    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76367ce6e9000061681340ed56ac8db0e3c2bcf664537b90382fc17f92f7770a"
Jul 18 01:55:12.277652 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.277627    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:12.316952 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.316923    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:12.383379 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.383347    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-util\") pod \"ea3b3415-162b-40bb-84e2-27afe590196b\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") "
Jul 18 01:55:12.383563 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.383405    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57lt8\" (UniqueName: \"kubernetes.io/projected/ea3b3415-162b-40bb-84e2-27afe590196b-kube-api-access-57lt8\") pod \"ea3b3415-162b-40bb-84e2-27afe590196b\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") "
Jul 18 01:55:12.383563 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.383485    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-bundle\") pod \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") "
Jul 18 01:55:12.383563 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.383511    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-util\") pod \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") "
Jul 18 01:55:12.383563 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.383535    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-bundle\") pod \"ea3b3415-162b-40bb-84e2-27afe590196b\" (UID: \"ea3b3415-162b-40bb-84e2-27afe590196b\") "
Jul 18 01:55:12.383777 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.383573    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6zbm\" (UniqueName: \"kubernetes.io/projected/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-kube-api-access-s6zbm\") pod \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\" (UID: \"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1\") "
Jul 18 01:55:12.384087 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.384057    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-bundle" (OuterVolumeSpecName: "bundle") pod "ea3b3415-162b-40bb-84e2-27afe590196b" (UID: "ea3b3415-162b-40bb-84e2-27afe590196b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:55:12.384206 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.384076    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-bundle" (OuterVolumeSpecName: "bundle") pod "a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" (UID: "a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:55:12.385738 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.385716    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea3b3415-162b-40bb-84e2-27afe590196b-kube-api-access-57lt8" (OuterVolumeSpecName: "kube-api-access-57lt8") pod "ea3b3415-162b-40bb-84e2-27afe590196b" (UID: "ea3b3415-162b-40bb-84e2-27afe590196b"). InnerVolumeSpecName "kube-api-access-57lt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:55:12.386191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.386161    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-kube-api-access-s6zbm" (OuterVolumeSpecName: "kube-api-access-s6zbm") pod "a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" (UID: "a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1"). InnerVolumeSpecName "kube-api-access-s6zbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:55:12.389316 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.389259    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-util" (OuterVolumeSpecName: "util") pod "a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" (UID: "a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:55:12.390106 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.390085    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-util" (OuterVolumeSpecName: "util") pod "ea3b3415-162b-40bb-84e2-27afe590196b" (UID: "ea3b3415-162b-40bb-84e2-27afe590196b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:55:12.484499 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.484463    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57lt8\" (UniqueName: \"kubernetes.io/projected/ea3b3415-162b-40bb-84e2-27afe590196b-kube-api-access-57lt8\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:12.484499 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.484496    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-bundle\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:12.484499 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.484505    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-util\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:12.484696 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.484512    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-bundle\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:12.484696 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.484520    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6zbm\" (UniqueName: \"kubernetes.io/projected/a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1-kube-api-access-s6zbm\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:12.484696 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:12.484528    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea3b3415-162b-40bb-84e2-27afe590196b-util\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:13.146612 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:13.146584    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f"
Jul 18 01:55:13.147063 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:13.146616    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f" event={"ID":"ea3b3415-162b-40bb-84e2-27afe590196b","Type":"ContainerDied","Data":"b775c9a9099987465e5613c9bf50b539e846e9de84e45003f91d0f46cf47ab51"}
Jul 18 01:55:13.147063 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:13.146668    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b775c9a9099987465e5613c9bf50b539e846e9de84e45003f91d0f46cf47ab51"
Jul 18 01:55:13.148648 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:13.148624    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk" event={"ID":"a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1","Type":"ContainerDied","Data":"c53a6d148f138f49ff1aa3d27758a05725026cb40fc0a5d3a431322dbd85bbdf"}
Jul 18 01:55:13.148791 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:13.148664    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c53a6d148f138f49ff1aa3d27758a05725026cb40fc0a5d3a431322dbd85bbdf"
Jul 18 01:55:13.148791 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:13.148683    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk"
Jul 18 01:55:19.842367 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:19.842331    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:23.271168 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:23.271118    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:24.525927 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.525893    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"]
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526216    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" containerName="extract"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526226    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" containerName="extract"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526236    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c33b59c-9f76-4033-88ca-9a2eab6b09ee" containerName="pull"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526241    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c33b59c-9f76-4033-88ca-9a2eab6b09ee" containerName="pull"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526248    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a0680e2-b156-4327-ae19-2ef7c9c508dd" containerName="pull"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526254    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a0680e2-b156-4327-ae19-2ef7c9c508dd" containerName="pull"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526263    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c33b59c-9f76-4033-88ca-9a2eab6b09ee" containerName="util"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526268    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c33b59c-9f76-4033-88ca-9a2eab6b09ee" containerName="util"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526274    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea3b3415-162b-40bb-84e2-27afe590196b" containerName="extract"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526279    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea3b3415-162b-40bb-84e2-27afe590196b" containerName="extract"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526288    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a0680e2-b156-4327-ae19-2ef7c9c508dd" containerName="util"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526293    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a0680e2-b156-4327-ae19-2ef7c9c508dd" containerName="util"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526299    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" containerName="pull"
Jul 18 01:55:24.526297 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526305    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" containerName="pull"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526314    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea3b3415-162b-40bb-84e2-27afe590196b" containerName="pull"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526319    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea3b3415-162b-40bb-84e2-27afe590196b" containerName="pull"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526325    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" containerName="util"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526330    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" containerName="util"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526335    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea3b3415-162b-40bb-84e2-27afe590196b" containerName="util"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526340    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea3b3415-162b-40bb-84e2-27afe590196b" containerName="util"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526350    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a0680e2-b156-4327-ae19-2ef7c9c508dd" containerName="extract"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526354    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a0680e2-b156-4327-ae19-2ef7c9c508dd" containerName="extract"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526360    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c33b59c-9f76-4033-88ca-9a2eab6b09ee" containerName="extract"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526364    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c33b59c-9f76-4033-88ca-9a2eab6b09ee" containerName="extract"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526414    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1" containerName="extract"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526438    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea3b3415-162b-40bb-84e2-27afe590196b" containerName="extract"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526447    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a0680e2-b156-4327-ae19-2ef7c9c508dd" containerName="extract"
Jul 18 01:55:24.526717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.526453    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c33b59c-9f76-4033-88ca-9a2eab6b09ee" containerName="extract"
Jul 18 01:55:24.530675 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.530658    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:24.532718 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.532693    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2b4bd\""
Jul 18 01:55:24.541123 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.541096    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"]
Jul 18 01:55:24.583378 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.583336    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" (UID: \"156f4a75-b4eb-45a9-86c3-dcb79bc6c554\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:24.583616 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.583450    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk2ml\" (UniqueName: \"kubernetes.io/projected/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-kube-api-access-wk2ml\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" (UID: \"156f4a75-b4eb-45a9-86c3-dcb79bc6c554\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:24.683826 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.683788    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" (UID: \"156f4a75-b4eb-45a9-86c3-dcb79bc6c554\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:24.683971 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.683864    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk2ml\" (UniqueName: \"kubernetes.io/projected/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-kube-api-access-wk2ml\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" (UID: \"156f4a75-b4eb-45a9-86c3-dcb79bc6c554\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:24.684237 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.684218    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" (UID: \"156f4a75-b4eb-45a9-86c3-dcb79bc6c554\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:24.695607 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.695585    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk2ml\" (UniqueName: \"kubernetes.io/projected/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-kube-api-access-wk2ml\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" (UID: \"156f4a75-b4eb-45a9-86c3-dcb79bc6c554\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:24.842067 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.841978    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:24.984690 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:24.984605    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"]
Jul 18 01:55:24.987352 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:24.987317    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod156f4a75_b4eb_45a9_86c3_dcb79bc6c554.slice/crio-134ee95a0a4aeb6350bde89ca425c0a06185bcc574e889dff88a6c94c31d6be6 WatchSource:0}: Error finding container 134ee95a0a4aeb6350bde89ca425c0a06185bcc574e889dff88a6c94c31d6be6: Status 404 returned error can't find the container with id 134ee95a0a4aeb6350bde89ca425c0a06185bcc574e889dff88a6c94c31d6be6
Jul 18 01:55:25.195769 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:25.195745    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:25.210093 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:25.210064    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28" event={"ID":"156f4a75-b4eb-45a9-86c3-dcb79bc6c554","Type":"ContainerStarted","Data":"134ee95a0a4aeb6350bde89ca425c0a06185bcc574e889dff88a6c94c31d6be6"}
Jul 18 01:55:26.546158 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:26.546122    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl"]
Jul 18 01:55:26.549487 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:26.549470    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl"
Jul 18 01:55:26.551871 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:26.551850    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 18 01:55:26.551988 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:26.551939    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qkxc4\""
Jul 18 01:55:26.567311 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:26.567289    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl"]
Jul 18 01:55:26.597796 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:26.597768    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bxms\" (UniqueName: \"kubernetes.io/projected/4fec861d-14a1-4f83-be98-f0f632875348-kube-api-access-7bxms\") pod \"dns-operator-controller-manager-864dd78c4c-s95xl\" (UID: \"4fec861d-14a1-4f83-be98-f0f632875348\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl"
Jul 18 01:55:26.699091 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:26.699039    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bxms\" (UniqueName: \"kubernetes.io/projected/4fec861d-14a1-4f83-be98-f0f632875348-kube-api-access-7bxms\") pod \"dns-operator-controller-manager-864dd78c4c-s95xl\" (UID: \"4fec861d-14a1-4f83-be98-f0f632875348\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl"
Jul 18 01:55:26.707809 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:26.707785    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bxms\" (UniqueName: \"kubernetes.io/projected/4fec861d-14a1-4f83-be98-f0f632875348-kube-api-access-7bxms\") pod \"dns-operator-controller-manager-864dd78c4c-s95xl\" (UID: \"4fec861d-14a1-4f83-be98-f0f632875348\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl"
Jul 18 01:55:26.861490 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:26.861392    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl"
Jul 18 01:55:27.018535 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:27.018501    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:27.019793 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:27.019775    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl"]
Jul 18 01:55:27.021979 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:27.021955    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fec861d_14a1_4f83_be98_f0f632875348.slice/crio-b9b40ea82d9c9b815b1a0425e31e2cfc4e9de97426d4346202b67f43dbab8c56 WatchSource:0}: Error finding container b9b40ea82d9c9b815b1a0425e31e2cfc4e9de97426d4346202b67f43dbab8c56: Status 404 returned error can't find the container with id b9b40ea82d9c9b815b1a0425e31e2cfc4e9de97426d4346202b67f43dbab8c56
Jul 18 01:55:27.167935 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:27.167901    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:27.221763 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:27.221728    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl" event={"ID":"4fec861d-14a1-4f83-be98-f0f632875348","Type":"ContainerStarted","Data":"b9b40ea82d9c9b815b1a0425e31e2cfc4e9de97426d4346202b67f43dbab8c56"}
Jul 18 01:55:29.378693 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:29.378658    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"]
Jul 18 01:55:29.383461 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:29.383443    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:29.386009 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:29.385990    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-j6dml\""
Jul 18 01:55:29.400620 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:29.400593    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"]
Jul 18 01:55:29.425945 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:29.425917    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw75m\" (UniqueName: \"kubernetes.io/projected/32633dbb-81bf-4769-b04a-69cde936e049-kube-api-access-fw75m\") pod \"limitador-operator-controller-manager-859d7b78bd-qnfr2\" (UID: \"32633dbb-81bf-4769-b04a-69cde936e049\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:29.442174 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:29.442146    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:29.526953 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:29.526918    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw75m\" (UniqueName: \"kubernetes.io/projected/32633dbb-81bf-4769-b04a-69cde936e049-kube-api-access-fw75m\") pod \"limitador-operator-controller-manager-859d7b78bd-qnfr2\" (UID: \"32633dbb-81bf-4769-b04a-69cde936e049\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:29.537118 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:29.537090    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw75m\" (UniqueName: \"kubernetes.io/projected/32633dbb-81bf-4769-b04a-69cde936e049-kube-api-access-fw75m\") pod \"limitador-operator-controller-manager-859d7b78bd-qnfr2\" (UID: \"32633dbb-81bf-4769-b04a-69cde936e049\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:29.575074 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:29.575031    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:29.694359 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:29.694325    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:30.019687 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:30.019661    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"]
Jul 18 01:55:30.021252 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:30.021230    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32633dbb_81bf_4769_b04a_69cde936e049.slice/crio-8472c6ff6cbd9cdd343aadd2d8f85b2b16648712c29a81bc605f08bd729afb57 WatchSource:0}: Error finding container 8472c6ff6cbd9cdd343aadd2d8f85b2b16648712c29a81bc605f08bd729afb57: Status 404 returned error can't find the container with id 8472c6ff6cbd9cdd343aadd2d8f85b2b16648712c29a81bc605f08bd729afb57
Jul 18 01:55:30.238287 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:30.238182    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl" event={"ID":"4fec861d-14a1-4f83-be98-f0f632875348","Type":"ContainerStarted","Data":"54fe9f7b282e49bb785ef52bbd75da4f85f4acbd3618bc1370d74536c2145d01"}
Jul 18 01:55:30.238287 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:30.238251    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl"
Jul 18 01:55:30.239536 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:30.239516    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2" event={"ID":"32633dbb-81bf-4769-b04a-69cde936e049","Type":"ContainerStarted","Data":"8472c6ff6cbd9cdd343aadd2d8f85b2b16648712c29a81bc605f08bd729afb57"}
Jul 18 01:55:30.241208 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:30.241182    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28" event={"ID":"156f4a75-b4eb-45a9-86c3-dcb79bc6c554","Type":"ContainerStarted","Data":"cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c"}
Jul 18 01:55:30.241336 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:30.241288    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:30.259437 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:30.259370    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl" podStartSLOduration=1.3565432610000001 podStartE2EDuration="4.259354467s" totalImagesPullingTime="2.902811206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:30.257235422 +0000 UTC m=+435.696732355" watchObservedRunningTime="2026-07-18 01:55:30.259354467 +0000 UTC m=+435.698851396"
Jul 18 01:55:30.274488 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:30.274404    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28" podStartSLOduration=1.394386014 podStartE2EDuration="6.274389255s" totalImagesPullingTime="4.880003241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:30.273493618 +0000 UTC m=+435.712990550" watchObservedRunningTime="2026-07-18 01:55:30.274389255 +0000 UTC m=+435.713886187"
Jul 18 01:55:30.406659 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:30.406618    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:31.806673 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:31.806587    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:31.902058 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:31.902021    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:32.254146 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.254104    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2" event={"ID":"32633dbb-81bf-4769-b04a-69cde936e049","Type":"ContainerStarted","Data":"212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da"}
Jul 18 01:55:32.254334 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.254193    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:32.270232 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.270188    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2" podStartSLOduration=1.804033333 podStartE2EDuration="3.270170457s" totalImagesPullingTime="1.466137124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:32.267740351 +0000 UTC m=+437.707237284" watchObservedRunningTime="2026-07-18 01:55:32.270170457 +0000 UTC m=+437.709667390"
Jul 18 01:55:32.757628 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.757593    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"]
Jul 18 01:55:32.761250 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.761233    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"
Jul 18 01:55:32.763266 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.763240    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 18 01:55:32.763414 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.763394    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 18 01:55:32.763544 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.763528    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qp2sp\""
Jul 18 01:55:32.769542 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.769518    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"]
Jul 18 01:55:32.778810 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.778783    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd"]
Jul 18 01:55:32.782218 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.782195    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd"
Jul 18 01:55:32.784727 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.784706    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-xj6qx\""
Jul 18 01:55:32.796988 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.796963    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd"]
Jul 18 01:55:32.801268 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.801239    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:32.857793 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.857759    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqgpf\" (UniqueName: \"kubernetes.io/projected/db37bfdb-41b4-4e6c-8b15-f15f709318d6-kube-api-access-mqgpf\") pod \"authorino-operator-75cc8d5cb7-5q2vd\" (UID: \"db37bfdb-41b4-4e6c-8b15-f15f709318d6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd"
Jul 18 01:55:32.858203 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.857805    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bbb6acd5-eb2e-4147-a131-bbb17b336305-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b9qrq\" (UID: \"bbb6acd5-eb2e-4147-a131-bbb17b336305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"
Jul 18 01:55:32.858203 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.857828    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbb6acd5-eb2e-4147-a131-bbb17b336305-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b9qrq\" (UID: \"bbb6acd5-eb2e-4147-a131-bbb17b336305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"
Jul 18 01:55:32.858203 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.857912    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7grl\" (UniqueName: \"kubernetes.io/projected/bbb6acd5-eb2e-4147-a131-bbb17b336305-kube-api-access-m7grl\") pod \"kuadrant-console-plugin-66b5774985-b9qrq\" (UID: \"bbb6acd5-eb2e-4147-a131-bbb17b336305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"
Jul 18 01:55:32.959379 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.959345    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bbb6acd5-eb2e-4147-a131-bbb17b336305-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b9qrq\" (UID: \"bbb6acd5-eb2e-4147-a131-bbb17b336305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"
Jul 18 01:55:32.959609 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.959386    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbb6acd5-eb2e-4147-a131-bbb17b336305-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b9qrq\" (UID: \"bbb6acd5-eb2e-4147-a131-bbb17b336305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"
Jul 18 01:55:32.959609 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.959468    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7grl\" (UniqueName: \"kubernetes.io/projected/bbb6acd5-eb2e-4147-a131-bbb17b336305-kube-api-access-m7grl\") pod \"kuadrant-console-plugin-66b5774985-b9qrq\" (UID: \"bbb6acd5-eb2e-4147-a131-bbb17b336305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"
Jul 18 01:55:32.959609 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.959519    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqgpf\" (UniqueName: \"kubernetes.io/projected/db37bfdb-41b4-4e6c-8b15-f15f709318d6-kube-api-access-mqgpf\") pod \"authorino-operator-75cc8d5cb7-5q2vd\" (UID: \"db37bfdb-41b4-4e6c-8b15-f15f709318d6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd"
Jul 18 01:55:32.960186 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.960166    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bbb6acd5-eb2e-4147-a131-bbb17b336305-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b9qrq\" (UID: \"bbb6acd5-eb2e-4147-a131-bbb17b336305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"
Jul 18 01:55:32.962056 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.962037    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbb6acd5-eb2e-4147-a131-bbb17b336305-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b9qrq\" (UID: \"bbb6acd5-eb2e-4147-a131-bbb17b336305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"
Jul 18 01:55:32.968009 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.967976    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqgpf\" (UniqueName: \"kubernetes.io/projected/db37bfdb-41b4-4e6c-8b15-f15f709318d6-kube-api-access-mqgpf\") pod \"authorino-operator-75cc8d5cb7-5q2vd\" (UID: \"db37bfdb-41b4-4e6c-8b15-f15f709318d6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd"
Jul 18 01:55:32.968892 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:32.968861    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7grl\" (UniqueName: \"kubernetes.io/projected/bbb6acd5-eb2e-4147-a131-bbb17b336305-kube-api-access-m7grl\") pod \"kuadrant-console-plugin-66b5774985-b9qrq\" (UID: \"bbb6acd5-eb2e-4147-a131-bbb17b336305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"
Jul 18 01:55:33.071311 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:33.071231    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"
Jul 18 01:55:33.092706 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:33.092676    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:33.093480 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:33.093455    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd"
Jul 18 01:55:33.229783 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:33.229758    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq"]
Jul 18 01:55:33.231650 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:33.231620    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbb6acd5_eb2e_4147_a131_bbb17b336305.slice/crio-212d9c96901ac55bb543f4905f493e52b45677f51c627aba40c9b04c8600b85f WatchSource:0}: Error finding container 212d9c96901ac55bb543f4905f493e52b45677f51c627aba40c9b04c8600b85f: Status 404 returned error can't find the container with id 212d9c96901ac55bb543f4905f493e52b45677f51c627aba40c9b04c8600b85f
Jul 18 01:55:33.247533 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:33.247509    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd"]
Jul 18 01:55:33.249656 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:33.249628    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb37bfdb_41b4_4e6c_8b15_f15f709318d6.slice/crio-f1e0a8c35689fdae93f3e1b1d2c410e7b82d9969e56a2afdf75d56e712513cf0 WatchSource:0}: Error finding container f1e0a8c35689fdae93f3e1b1d2c410e7b82d9969e56a2afdf75d56e712513cf0: Status 404 returned error can't find the container with id f1e0a8c35689fdae93f3e1b1d2c410e7b82d9969e56a2afdf75d56e712513cf0
Jul 18 01:55:33.260857 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:33.260825    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd" event={"ID":"db37bfdb-41b4-4e6c-8b15-f15f709318d6","Type":"ContainerStarted","Data":"f1e0a8c35689fdae93f3e1b1d2c410e7b82d9969e56a2afdf75d56e712513cf0"}
Jul 18 01:55:33.262098 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:33.262073    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq" event={"ID":"bbb6acd5-eb2e-4147-a131-bbb17b336305","Type":"ContainerStarted","Data":"212d9c96901ac55bb543f4905f493e52b45677f51c627aba40c9b04c8600b85f"}
Jul 18 01:55:34.764738 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:34.764642    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:34.832136 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:34.832062    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:38.303234 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:38.303193    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd" event={"ID":"db37bfdb-41b4-4e6c-8b15-f15f709318d6","Type":"ContainerStarted","Data":"8384722221e8e5082ba861f631f7842768a08ee73fd164366e90d3a132676e4c"}
Jul 18 01:55:38.303704 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:38.303364    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd"
Jul 18 01:55:38.318349 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:38.318303    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd" podStartSLOduration=1.911161426 podStartE2EDuration="6.318287678s" totalImagesPullingTime="4.407126252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:38.317282461 +0000 UTC m=+443.756779394" watchObservedRunningTime="2026-07-18 01:55:38.318287678 +0000 UTC m=+443.757784610"
Jul 18 01:55:41.249962 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:41.249933    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:41.250479 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:41.250012    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s95xl"
Jul 18 01:55:42.598182 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:42.598142    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:42.699849 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:42.699815    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:43.134246 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.134203    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"]
Jul 18 01:55:43.134562 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.134499    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" containerName="manager" containerID="cri-o://cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c" gracePeriod=2
Jul 18 01:55:43.137621 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:43.137579    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:43.137749 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.137656    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"]
Jul 18 01:55:43.150052 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.150024    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"]
Jul 18 01:55:43.150383 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.150339    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2" podUID="32633dbb-81bf-4769-b04a-69cde936e049" containerName="manager" containerID="cri-o://212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da" gracePeriod=2
Jul 18 01:55:43.152685 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:43.152652    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:43.153040 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.153013    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:43.162702 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.160099    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"]
Jul 18 01:55:43.162702 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:43.162103    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:43.162702 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.162465    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"]
Jul 18 01:55:43.162979 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.162921    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" containerName="manager"
Jul 18 01:55:43.162979 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.162934    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" containerName="manager"
Jul 18 01:55:43.162979 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.162954    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32633dbb-81bf-4769-b04a-69cde936e049" containerName="manager"
Jul 18 01:55:43.162979 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.162959    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="32633dbb-81bf-4769-b04a-69cde936e049" containerName="manager"
Jul 18 01:55:43.163196 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.163043    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="32633dbb-81bf-4769-b04a-69cde936e049" containerName="manager"
Jul 18 01:55:43.163196 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.163056    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" containerName="manager"
Jul 18 01:55:43.163637 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:43.163593    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnfr2\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="32633dbb-81bf-4769-b04a-69cde936e049" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:43.181515 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.181477    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4"]
Jul 18 01:55:43.181752 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.181729    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:55:43.185798 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.185775    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"]
Jul 18 01:55:43.185896 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.185884    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4"
Jul 18 01:55:43.186469 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:43.186415    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:43.187530 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.187288    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4"]
Jul 18 01:55:43.188336 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:43.188306    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnfr2\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="32633dbb-81bf-4769-b04a-69cde936e049" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:43.215291 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:43.214672    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:43.216275 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:43.216242    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnfr2\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="32633dbb-81bf-4769-b04a-69cde936e049" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:43.259166 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.259123    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svvb6\" (UniqueName: \"kubernetes.io/projected/3f6d5aee-235d-4030-af81-bbea6309c44b-kube-api-access-svvb6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sh8k2\" (UID: \"3f6d5aee-235d-4030-af81-bbea6309c44b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:55:43.259365 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.259232    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f6d5aee-235d-4030-af81-bbea6309c44b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sh8k2\" (UID: \"3f6d5aee-235d-4030-af81-bbea6309c44b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:55:43.361001 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.360731    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svvb6\" (UniqueName: \"kubernetes.io/projected/3f6d5aee-235d-4030-af81-bbea6309c44b-kube-api-access-svvb6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sh8k2\" (UID: \"3f6d5aee-235d-4030-af81-bbea6309c44b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:55:43.361001 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.360782    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mnl9\" (UniqueName: \"kubernetes.io/projected/5efce080-fe0d-4a6b-b834-b2993e4a6d53-kube-api-access-7mnl9\") pod \"limitador-operator-controller-manager-859d7b78bd-254h4\" (UID: \"5efce080-fe0d-4a6b-b834-b2993e4a6d53\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4"
Jul 18 01:55:43.361001 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.360842    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f6d5aee-235d-4030-af81-bbea6309c44b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sh8k2\" (UID: \"3f6d5aee-235d-4030-af81-bbea6309c44b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:55:43.361267 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.361211    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f6d5aee-235d-4030-af81-bbea6309c44b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sh8k2\" (UID: \"3f6d5aee-235d-4030-af81-bbea6309c44b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:55:43.369882 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.369851    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svvb6\" (UniqueName: \"kubernetes.io/projected/3f6d5aee-235d-4030-af81-bbea6309c44b-kube-api-access-svvb6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sh8k2\" (UID: \"3f6d5aee-235d-4030-af81-bbea6309c44b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:55:43.462222 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.462176    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mnl9\" (UniqueName: \"kubernetes.io/projected/5efce080-fe0d-4a6b-b834-b2993e4a6d53-kube-api-access-7mnl9\") pod \"limitador-operator-controller-manager-859d7b78bd-254h4\" (UID: \"5efce080-fe0d-4a6b-b834-b2993e4a6d53\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4"
Jul 18 01:55:43.470463 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.470410    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mnl9\" (UniqueName: \"kubernetes.io/projected/5efce080-fe0d-4a6b-b834-b2993e4a6d53-kube-api-access-7mnl9\") pod \"limitador-operator-controller-manager-859d7b78bd-254h4\" (UID: \"5efce080-fe0d-4a6b-b834-b2993e4a6d53\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4"
Jul 18 01:55:43.596903 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.596863    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:55:43.604001 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.603970    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4"
Jul 18 01:55:43.938338 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.938292    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"]
Jul 18 01:55:43.955192 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.955160    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 01:55:43.956745 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:43.956711    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"]
Jul 18 01:55:43.985837 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:43.985792    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:43.987478 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:43.987447    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnfr2\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="32633dbb-81bf-4769-b04a-69cde936e049" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:44.067417 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:44.067383    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/043e2277-2848-47ca-9564-9b2adec4114e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7cz7g\" (UID: \"043e2277-2848-47ca-9564-9b2adec4114e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 01:55:44.067644 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:44.067466    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dqx6\" (UniqueName: \"kubernetes.io/projected/043e2277-2848-47ca-9564-9b2adec4114e-kube-api-access-6dqx6\") pod \"kuadrant-operator-controller-manager-86878c769f-7cz7g\" (UID: \"043e2277-2848-47ca-9564-9b2adec4114e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 01:55:44.168969 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:44.168937    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/043e2277-2848-47ca-9564-9b2adec4114e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7cz7g\" (UID: \"043e2277-2848-47ca-9564-9b2adec4114e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 01:55:44.169159 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:44.168996    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dqx6\" (UniqueName: \"kubernetes.io/projected/043e2277-2848-47ca-9564-9b2adec4114e-kube-api-access-6dqx6\") pod \"kuadrant-operator-controller-manager-86878c769f-7cz7g\" (UID: \"043e2277-2848-47ca-9564-9b2adec4114e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 01:55:44.169349 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:44.169320    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/043e2277-2848-47ca-9564-9b2adec4114e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7cz7g\" (UID: \"043e2277-2848-47ca-9564-9b2adec4114e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 01:55:44.177314 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:44.177276    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dqx6\" (UniqueName: \"kubernetes.io/projected/043e2277-2848-47ca-9564-9b2adec4114e-kube-api-access-6dqx6\") pod \"kuadrant-operator-controller-manager-86878c769f-7cz7g\" (UID: \"043e2277-2848-47ca-9564-9b2adec4114e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 01:55:44.269126 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:44.269032    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 01:55:45.075288 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:45.074679    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n5q28\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:45.076338 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:45.076302    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnfr2\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="32633dbb-81bf-4769-b04a-69cde936e049" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:47.202081 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:47.202047    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:47.302916 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:47.302878    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:49.313395 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:49.313365    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5q2vd"
Jul 18 01:55:51.987443 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:51.987391    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:52.087887 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:52.087845    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:54.633386 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.632637    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:54.658268 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.658098    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:54.665395 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.665368    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk2ml\" (UniqueName: \"kubernetes.io/projected/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-kube-api-access-wk2ml\") pod \"156f4a75-b4eb-45a9-86c3-dcb79bc6c554\" (UID: \"156f4a75-b4eb-45a9-86c3-dcb79bc6c554\") "
Jul 18 01:55:54.665521 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.665472    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw75m\" (UniqueName: \"kubernetes.io/projected/32633dbb-81bf-4769-b04a-69cde936e049-kube-api-access-fw75m\") pod \"32633dbb-81bf-4769-b04a-69cde936e049\" (UID: \"32633dbb-81bf-4769-b04a-69cde936e049\") "
Jul 18 01:55:54.665571 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.665534    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-extensions-socket-volume\") pod \"156f4a75-b4eb-45a9-86c3-dcb79bc6c554\" (UID: \"156f4a75-b4eb-45a9-86c3-dcb79bc6c554\") "
Jul 18 01:55:54.666137 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.666112    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "156f4a75-b4eb-45a9-86c3-dcb79bc6c554" (UID: "156f4a75-b4eb-45a9-86c3-dcb79bc6c554"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:55:54.668868 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.668830    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32633dbb-81bf-4769-b04a-69cde936e049-kube-api-access-fw75m" (OuterVolumeSpecName: "kube-api-access-fw75m") pod "32633dbb-81bf-4769-b04a-69cde936e049" (UID: "32633dbb-81bf-4769-b04a-69cde936e049"). InnerVolumeSpecName "kube-api-access-fw75m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:55:54.668961 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.668881    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-kube-api-access-wk2ml" (OuterVolumeSpecName: "kube-api-access-wk2ml") pod "156f4a75-b4eb-45a9-86c3-dcb79bc6c554" (UID: "156f4a75-b4eb-45a9-86c3-dcb79bc6c554"). InnerVolumeSpecName "kube-api-access-wk2ml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:55:54.682214 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.682190    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4"]
Jul 18 01:55:54.685490 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:54.685450    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5efce080_fe0d_4a6b_b834_b2993e4a6d53.slice/crio-e3c5d30e639c1895f7de6f78d3b487cc23d4560935b9ff8c8877af419c4e7105 WatchSource:0}: Error finding container e3c5d30e639c1895f7de6f78d3b487cc23d4560935b9ff8c8877af419c4e7105: Status 404 returned error can't find the container with id e3c5d30e639c1895f7de6f78d3b487cc23d4560935b9ff8c8877af419c4e7105
Jul 18 01:55:54.717306 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.717137    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"]
Jul 18 01:55:54.718006 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:54.717940    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod043e2277_2848_47ca_9564_9b2adec4114e.slice/crio-deb00cce0fc196428fd5b5be4ae9d8ac47c2203876cfa0102b998d8ae29a95d1 WatchSource:0}: Error finding container deb00cce0fc196428fd5b5be4ae9d8ac47c2203876cfa0102b998d8ae29a95d1: Status 404 returned error can't find the container with id deb00cce0fc196428fd5b5be4ae9d8ac47c2203876cfa0102b998d8ae29a95d1
Jul 18 01:55:54.742191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.741982    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"]
Jul 18 01:55:54.745969 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:55:54.745948    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6d5aee_235d_4030_af81_bbea6309c44b.slice/crio-fe8205bc6b21a17cc71c747f8a6c32b99f63da2c994ba4c75d0055d6e4d336f5 WatchSource:0}: Error finding container fe8205bc6b21a17cc71c747f8a6c32b99f63da2c994ba4c75d0055d6e4d336f5: Status 404 returned error can't find the container with id fe8205bc6b21a17cc71c747f8a6c32b99f63da2c994ba4c75d0055d6e4d336f5
Jul 18 01:55:54.766351 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.766324    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw75m\" (UniqueName: \"kubernetes.io/projected/32633dbb-81bf-4769-b04a-69cde936e049-kube-api-access-fw75m\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:54.766351 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.766353    2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-extensions-socket-volume\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:54.766520 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:54.766364    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk2ml\" (UniqueName: \"kubernetes.io/projected/156f4a75-b4eb-45a9-86c3-dcb79bc6c554-kube-api-access-wk2ml\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:55:55.076192 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.076094    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" path="/var/lib/kubelet/pods/156f4a75-b4eb-45a9-86c3-dcb79bc6c554/volumes"
Jul 18 01:55:55.076549 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.076528    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32633dbb-81bf-4769-b04a-69cde936e049" path="/var/lib/kubelet/pods/32633dbb-81bf-4769-b04a-69cde936e049/volumes"
Jul 18 01:55:55.418902 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.418869    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4" event={"ID":"5efce080-fe0d-4a6b-b834-b2993e4a6d53","Type":"ContainerStarted","Data":"294acbaca9447d80d3f774102480ecfb5732026f0ecbe2b13bb8338e0828d5e9"}
Jul 18 01:55:55.419083 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.418920    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4" event={"ID":"5efce080-fe0d-4a6b-b834-b2993e4a6d53","Type":"ContainerStarted","Data":"e3c5d30e639c1895f7de6f78d3b487cc23d4560935b9ff8c8877af419c4e7105"}
Jul 18 01:55:55.419083 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.418967    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4"
Jul 18 01:55:55.420483 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.420455    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq" event={"ID":"bbb6acd5-eb2e-4147-a131-bbb17b336305","Type":"ContainerStarted","Data":"2dc86e905f2a34b6ded9c8c9ce16656397c86c245177a47ba2b56eed31c19db0"}
Jul 18 01:55:55.421935 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.421910    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g" event={"ID":"043e2277-2848-47ca-9564-9b2adec4114e","Type":"ContainerStarted","Data":"afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5"}
Jul 18 01:55:55.422049 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.421962    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g" event={"ID":"043e2277-2848-47ca-9564-9b2adec4114e","Type":"ContainerStarted","Data":"deb00cce0fc196428fd5b5be4ae9d8ac47c2203876cfa0102b998d8ae29a95d1"}
Jul 18 01:55:55.422049 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.421982    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 01:55:55.423114 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.423095    2552 generic.go:356] "Generic (PLEG): container finished" podID="32633dbb-81bf-4769-b04a-69cde936e049" containerID="212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da" exitCode=0
Jul 18 01:55:55.423223 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.423127    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnfr2"
Jul 18 01:55:55.423223 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.423157    2552 scope.go:117] "RemoveContainer" containerID="212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da"
Jul 18 01:55:55.424898 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.424878    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2" event={"ID":"3f6d5aee-235d-4030-af81-bbea6309c44b","Type":"ContainerStarted","Data":"e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6"}
Jul 18 01:55:55.425006 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.424920    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2" event={"ID":"3f6d5aee-235d-4030-af81-bbea6309c44b","Type":"ContainerStarted","Data":"fe8205bc6b21a17cc71c747f8a6c32b99f63da2c994ba4c75d0055d6e4d336f5"}
Jul 18 01:55:55.425006 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.424948    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:55:55.426370 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.426351    2552 generic.go:356] "Generic (PLEG): container finished" podID="156f4a75-b4eb-45a9-86c3-dcb79bc6c554" containerID="cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c" exitCode=0
Jul 18 01:55:55.426466 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.426383    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n5q28"
Jul 18 01:55:55.433157 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.433142    2552 scope.go:117] "RemoveContainer" containerID="212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da"
Jul 18 01:55:55.433401 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:55.433383    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da\": container with ID starting with 212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da not found: ID does not exist" containerID="212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da"
Jul 18 01:55:55.433486 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.433408    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da"} err="failed to get container status \"212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da\": rpc error: code = NotFound desc = could not find container \"212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da\": container with ID starting with 212fe0347d5883ee66b8dca52570990622a32ecc4a80fb90e840a8f8753f38da not found: ID does not exist"
Jul 18 01:55:55.433486 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.433442    2552 scope.go:117] "RemoveContainer" containerID="cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c"
Jul 18 01:55:55.435709 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.435674    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4" podStartSLOduration=12.435661468 podStartE2EDuration="12.435661468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:55.434488419 +0000 UTC m=+460.873985364" watchObservedRunningTime="2026-07-18 01:55:55.435661468 +0000 UTC m=+460.875158378"
Jul 18 01:55:55.442256 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.442236    2552 scope.go:117] "RemoveContainer" containerID="cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c"
Jul 18 01:55:55.442568 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:55:55.442540    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c\": container with ID starting with cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c not found: ID does not exist" containerID="cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c"
Jul 18 01:55:55.442634 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.442575    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c"} err="failed to get container status \"cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c\": rpc error: code = NotFound desc = could not find container \"cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c\": container with ID starting with cfcd3b7384481db04c2ebff950efa36c27dcf414aaf632d0637475f6fa33ee8c not found: ID does not exist"
Jul 18 01:55:55.455072 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.455031    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2" podStartSLOduration=12.455018324 podStartE2EDuration="12.455018324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:55.452651599 +0000 UTC m=+460.892148531" watchObservedRunningTime="2026-07-18 01:55:55.455018324 +0000 UTC m=+460.894515313"
Jul 18 01:55:55.472612 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.472568    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g" podStartSLOduration=12.472556684 podStartE2EDuration="12.472556684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:55.471111606 +0000 UTC m=+460.910608556" watchObservedRunningTime="2026-07-18 01:55:55.472556684 +0000 UTC m=+460.912053616"
Jul 18 01:55:55.486838 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:55.486795    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b9qrq" podStartSLOduration=2.162973554 podStartE2EDuration="23.48678098s" totalImagesPullingTime="21.323807426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:55.485661314 +0000 UTC m=+460.925158247" watchObservedRunningTime="2026-07-18 01:55:55.48678098 +0000 UTC m=+460.926277912"
Jul 18 01:55:56.220703 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:56.220672    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:55:56.286625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:55:56.286595    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:00.220409 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:00.220376    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:00.313202 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:00.313165    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:03.384185 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:03.384148    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:03.444955 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:03.444923    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:06.435675 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:06.435643    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 01:56:06.436079 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:06.435720    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:56:06.436079 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:06.435755    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-254h4"
Jul 18 01:56:06.561729 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:06.561693    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"]
Jul 18 01:56:06.562007 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:06.561981    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2" podUID="3f6d5aee-235d-4030-af81-bbea6309c44b" containerName="manager" containerID="cri-o://e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6" gracePeriod=10
Jul 18 01:56:06.854777 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:06.854741    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:56:06.969304 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:06.969203    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svvb6\" (UniqueName: \"kubernetes.io/projected/3f6d5aee-235d-4030-af81-bbea6309c44b-kube-api-access-svvb6\") pod \"3f6d5aee-235d-4030-af81-bbea6309c44b\" (UID: \"3f6d5aee-235d-4030-af81-bbea6309c44b\") "
Jul 18 01:56:06.969516 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:06.969304    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f6d5aee-235d-4030-af81-bbea6309c44b-extensions-socket-volume\") pod \"3f6d5aee-235d-4030-af81-bbea6309c44b\" (UID: \"3f6d5aee-235d-4030-af81-bbea6309c44b\") "
Jul 18 01:56:06.969743 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:06.969710    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f6d5aee-235d-4030-af81-bbea6309c44b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3f6d5aee-235d-4030-af81-bbea6309c44b" (UID: "3f6d5aee-235d-4030-af81-bbea6309c44b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:56:06.971635 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:06.971608    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f6d5aee-235d-4030-af81-bbea6309c44b-kube-api-access-svvb6" (OuterVolumeSpecName: "kube-api-access-svvb6") pod "3f6d5aee-235d-4030-af81-bbea6309c44b" (UID: "3f6d5aee-235d-4030-af81-bbea6309c44b"). InnerVolumeSpecName "kube-api-access-svvb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:07.069916 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.069884    2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f6d5aee-235d-4030-af81-bbea6309c44b-extensions-socket-volume\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:56:07.069916 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.069912    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svvb6\" (UniqueName: \"kubernetes.io/projected/3f6d5aee-235d-4030-af81-bbea6309c44b-kube-api-access-svvb6\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:56:07.493502 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.493468    2552 generic.go:356] "Generic (PLEG): container finished" podID="3f6d5aee-235d-4030-af81-bbea6309c44b" containerID="e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6" exitCode=0
Jul 18 01:56:07.493957 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.493514    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"
Jul 18 01:56:07.493957 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.493506    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2" event={"ID":"3f6d5aee-235d-4030-af81-bbea6309c44b","Type":"ContainerDied","Data":"e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6"}
Jul 18 01:56:07.493957 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.493652    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2" event={"ID":"3f6d5aee-235d-4030-af81-bbea6309c44b","Type":"ContainerDied","Data":"fe8205bc6b21a17cc71c747f8a6c32b99f63da2c994ba4c75d0055d6e4d336f5"}
Jul 18 01:56:07.493957 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.493687    2552 scope.go:117] "RemoveContainer" containerID="e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6"
Jul 18 01:56:07.502560 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.502544    2552 scope.go:117] "RemoveContainer" containerID="e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6"
Jul 18 01:56:07.502819 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:56:07.502802    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6\": container with ID starting with e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6 not found: ID does not exist" containerID="e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6"
Jul 18 01:56:07.502859 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.502832    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6"} err="failed to get container status \"e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6\": rpc error: code = NotFound desc = could not find container \"e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6\": container with ID starting with e950cb054e161e501f591bf7e726d06a53f7b8ea769aabe5b2bc0caf42afa1d6 not found: ID does not exist"
Jul 18 01:56:07.521310 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.521283    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"]
Jul 18 01:56:07.523462 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.523442    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sh8k2"]
Jul 18 01:56:07.610673 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.610638    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:07.710954 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:07.710923    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:09.076095 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:09.076054    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f6d5aee-235d-4030-af81-bbea6309c44b" path="/var/lib/kubelet/pods/3f6d5aee-235d-4030-af81-bbea6309c44b/volumes"
Jul 18 01:56:11.482553 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:11.482518    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:11.556665 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:11.556619    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:12.418262 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:12.418224    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:12.585597 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:12.585569    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:23.073483 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:23.073447    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:23.139458 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:23.139405    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:24.855043 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:24.855012    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"]
Jul 18 01:56:24.855412 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:24.855356    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f6d5aee-235d-4030-af81-bbea6309c44b" containerName="manager"
Jul 18 01:56:24.855412 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:24.855367    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f6d5aee-235d-4030-af81-bbea6309c44b" containerName="manager"
Jul 18 01:56:24.855514 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:24.855460    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f6d5aee-235d-4030-af81-bbea6309c44b" containerName="manager"
Jul 18 01:56:24.866100 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:24.866065    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:24.914521 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:24.914487    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"]
Jul 18 01:56:24.914705 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:24.914603    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:24.916718 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:24.916697    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 18 01:56:24.957040 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:24.957008    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"]
Jul 18 01:56:24.964184 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:24.964157    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:25.022280 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.022249    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzq5l\" (UniqueName: \"kubernetes.io/projected/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-kube-api-access-gzq5l\") pod \"limitador-limitador-64c94fd64f-kr4qj\" (UID: \"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:25.022473 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.022304    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-config-file\") pod \"limitador-limitador-64c94fd64f-kr4qj\" (UID: \"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:25.063191 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.063160    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:25.123586 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.123494    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-config-file\") pod \"limitador-limitador-64c94fd64f-kr4qj\" (UID: \"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:25.123762 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.123658    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzq5l\" (UniqueName: \"kubernetes.io/projected/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-kube-api-access-gzq5l\") pod \"limitador-limitador-64c94fd64f-kr4qj\" (UID: \"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:25.124158 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.124140    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-config-file\") pod \"limitador-limitador-64c94fd64f-kr4qj\" (UID: \"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:25.132745 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.132721    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzq5l\" (UniqueName: \"kubernetes.io/projected/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-kube-api-access-gzq5l\") pod \"limitador-limitador-64c94fd64f-kr4qj\" (UID: \"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:25.227544 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.227511    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:25.382668 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.382638    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"]
Jul 18 01:56:25.385739 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:56:25.385713    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d1ee3e_8bc9_4910_8e37_015fb16d5e2d.slice/crio-6113bdbaf0375306fbf39683bcfbc4ab8dd4e5999bb50f36bc86be347b500412 WatchSource:0}: Error finding container 6113bdbaf0375306fbf39683bcfbc4ab8dd4e5999bb50f36bc86be347b500412: Status 404 returned error can't find the container with id 6113bdbaf0375306fbf39683bcfbc4ab8dd4e5999bb50f36bc86be347b500412
Jul 18 01:56:25.458603 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.458570    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:25.548785 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.548748    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:25.594059 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.594028    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj" event={"ID":"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d","Type":"ContainerStarted","Data":"6113bdbaf0375306fbf39683bcfbc4ab8dd4e5999bb50f36bc86be347b500412"}
Jul 18 01:56:25.713612 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.713580    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-4ltkn"]
Jul 18 01:56:25.718354 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.718333    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4ltkn"
Jul 18 01:56:25.720701 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.720615    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zklft\""
Jul 18 01:56:25.723754 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.723694    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4ltkn"]
Jul 18 01:56:25.743314 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.743258    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:25.830567 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.830531    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh8gp\" (UniqueName: \"kubernetes.io/projected/f1f97f22-d7e9-4c27-8c91-c07490fa408c-kube-api-access-qh8gp\") pod \"authorino-58cd955675-4ltkn\" (UID: \"f1f97f22-d7e9-4c27-8c91-c07490fa408c\") " pod="kuadrant-system/authorino-58cd955675-4ltkn"
Jul 18 01:56:25.844090 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.844043    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:25.931323 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.931282    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh8gp\" (UniqueName: \"kubernetes.io/projected/f1f97f22-d7e9-4c27-8c91-c07490fa408c-kube-api-access-qh8gp\") pod \"authorino-58cd955675-4ltkn\" (UID: \"f1f97f22-d7e9-4c27-8c91-c07490fa408c\") " pod="kuadrant-system/authorino-58cd955675-4ltkn"
Jul 18 01:56:25.943681 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:25.941576    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh8gp\" (UniqueName: \"kubernetes.io/projected/f1f97f22-d7e9-4c27-8c91-c07490fa408c-kube-api-access-qh8gp\") pod \"authorino-58cd955675-4ltkn\" (UID: \"f1f97f22-d7e9-4c27-8c91-c07490fa408c\") " pod="kuadrant-system/authorino-58cd955675-4ltkn"
Jul 18 01:56:26.030107 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:26.030015    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4ltkn"
Jul 18 01:56:26.219054 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:26.218967    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4ltkn"]
Jul 18 01:56:26.222175 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:56:26.222141    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1f97f22_d7e9_4c27_8c91_c07490fa408c.slice/crio-d77ef9dce767d736ee705adfac810987ff20594e6a53c3c9cbc80007a09161cd WatchSource:0}: Error finding container d77ef9dce767d736ee705adfac810987ff20594e6a53c3c9cbc80007a09161cd: Status 404 returned error can't find the container with id d77ef9dce767d736ee705adfac810987ff20594e6a53c3c9cbc80007a09161cd
Jul 18 01:56:26.600490 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:26.600457    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4ltkn" event={"ID":"f1f97f22-d7e9-4c27-8c91-c07490fa408c","Type":"ContainerStarted","Data":"d77ef9dce767d736ee705adfac810987ff20594e6a53c3c9cbc80007a09161cd"}
Jul 18 01:56:28.618362 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:28.618161    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj" event={"ID":"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d","Type":"ContainerStarted","Data":"df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87"}
Jul 18 01:56:28.618928 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:28.618468    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:28.637824 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:28.637764    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj" podStartSLOduration=1.8467816529999999 podStartE2EDuration="4.637743564s" totalImagesPullingTime="2.790961911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:28.633954811 +0000 UTC m=+494.073451766" watchObservedRunningTime="2026-07-18 01:56:28.637743564 +0000 UTC m=+494.077240497"
Jul 18 01:56:29.866663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:29.866631    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4ltkn"]
Jul 18 01:56:30.631715 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:30.631679    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4ltkn" event={"ID":"f1f97f22-d7e9-4c27-8c91-c07490fa408c","Type":"ContainerStarted","Data":"9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a"}
Jul 18 01:56:30.631715 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:30.631691    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-4ltkn" podUID="f1f97f22-d7e9-4c27-8c91-c07490fa408c" containerName="authorino" containerID="cri-o://9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a" gracePeriod=30
Jul 18 01:56:30.647812 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:30.647768    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-4ltkn" podStartSLOduration=1.842527299 podStartE2EDuration="5.647751998s" totalImagesPullingTime="3.805224699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:30.64535629 +0000 UTC m=+496.084853226" watchObservedRunningTime="2026-07-18 01:56:30.647751998 +0000 UTC m=+496.087248930"
Jul 18 01:56:30.884094 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:30.884034    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4ltkn"
Jul 18 01:56:30.979188 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:30.979155    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh8gp\" (UniqueName: \"kubernetes.io/projected/f1f97f22-d7e9-4c27-8c91-c07490fa408c-kube-api-access-qh8gp\") pod \"f1f97f22-d7e9-4c27-8c91-c07490fa408c\" (UID: \"f1f97f22-d7e9-4c27-8c91-c07490fa408c\") "
Jul 18 01:56:30.981345 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:30.981314    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f97f22-d7e9-4c27-8c91-c07490fa408c-kube-api-access-qh8gp" (OuterVolumeSpecName: "kube-api-access-qh8gp") pod "f1f97f22-d7e9-4c27-8c91-c07490fa408c" (UID: "f1f97f22-d7e9-4c27-8c91-c07490fa408c"). InnerVolumeSpecName "kube-api-access-qh8gp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:31.083256 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:31.083216    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh8gp\" (UniqueName: \"kubernetes.io/projected/f1f97f22-d7e9-4c27-8c91-c07490fa408c-kube-api-access-qh8gp\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:56:31.637784 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:31.637751    2552 generic.go:356] "Generic (PLEG): container finished" podID="f1f97f22-d7e9-4c27-8c91-c07490fa408c" containerID="9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a" exitCode=0
Jul 18 01:56:31.637995 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:31.637820    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4ltkn"
Jul 18 01:56:31.637995 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:31.637830    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4ltkn" event={"ID":"f1f97f22-d7e9-4c27-8c91-c07490fa408c","Type":"ContainerDied","Data":"9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a"}
Jul 18 01:56:31.637995 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:31.637873    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4ltkn" event={"ID":"f1f97f22-d7e9-4c27-8c91-c07490fa408c","Type":"ContainerDied","Data":"d77ef9dce767d736ee705adfac810987ff20594e6a53c3c9cbc80007a09161cd"}
Jul 18 01:56:31.637995 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:31.637891    2552 scope.go:117] "RemoveContainer" containerID="9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a"
Jul 18 01:56:31.646750 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:31.646729    2552 scope.go:117] "RemoveContainer" containerID="9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a"
Jul 18 01:56:31.646999 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:56:31.646974    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a\": container with ID starting with 9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a not found: ID does not exist" containerID="9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a"
Jul 18 01:56:31.647048 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:31.647008    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a"} err="failed to get container status \"9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a\": rpc error: code = NotFound desc = could not find container \"9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a\": container with ID starting with 9e95e321200ca0611e270beca42d72c2f619b3c8f96e721e05a8a887fdf2767a not found: ID does not exist"
Jul 18 01:56:31.652633 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:31.652610    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4ltkn"]
Jul 18 01:56:31.655882 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:31.655864    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-4ltkn"]
Jul 18 01:56:33.077134 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:33.077097    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1f97f22-d7e9-4c27-8c91-c07490fa408c" path="/var/lib/kubelet/pods/f1f97f22-d7e9-4c27-8c91-c07490fa408c/volumes"
Jul 18 01:56:36.048697 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:36.048671    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:36.150832 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:36.150791    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:36.211127 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:36.211096    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:39.420287 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:39.420253    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"]
Jul 18 01:56:39.421157 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:39.421122    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj" podUID="72d1ee3e-8bc9-4910-8e37-015fb16d5e2d" containerName="limitador" containerID="cri-o://df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87" gracePeriod=30
Jul 18 01:56:39.421795 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:39.421764    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:39.966185 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:39.966163    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:40.059653 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.059565    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-config-file\") pod \"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d\" (UID: \"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d\") "
Jul 18 01:56:40.059653 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.059615    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzq5l\" (UniqueName: \"kubernetes.io/projected/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-kube-api-access-gzq5l\") pod \"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d\" (UID: \"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d\") "
Jul 18 01:56:40.059943 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.059922    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-config-file" (OuterVolumeSpecName: "config-file") pod "72d1ee3e-8bc9-4910-8e37-015fb16d5e2d" (UID: "72d1ee3e-8bc9-4910-8e37-015fb16d5e2d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:40.061795 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.061770    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-kube-api-access-gzq5l" (OuterVolumeSpecName: "kube-api-access-gzq5l") pod "72d1ee3e-8bc9-4910-8e37-015fb16d5e2d" (UID: "72d1ee3e-8bc9-4910-8e37-015fb16d5e2d"). InnerVolumeSpecName "kube-api-access-gzq5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:40.160560 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.160523    2552 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-config-file\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:56:40.160560 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.160555    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzq5l\" (UniqueName: \"kubernetes.io/projected/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d-kube-api-access-gzq5l\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:56:40.685900 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.685865    2552 generic.go:356] "Generic (PLEG): container finished" podID="72d1ee3e-8bc9-4910-8e37-015fb16d5e2d" containerID="df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87" exitCode=0
Jul 18 01:56:40.686376 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.685918    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"
Jul 18 01:56:40.686376 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.685959    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj" event={"ID":"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d","Type":"ContainerDied","Data":"df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87"}
Jul 18 01:56:40.686376 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.686013    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kr4qj" event={"ID":"72d1ee3e-8bc9-4910-8e37-015fb16d5e2d","Type":"ContainerDied","Data":"6113bdbaf0375306fbf39683bcfbc4ab8dd4e5999bb50f36bc86be347b500412"}
Jul 18 01:56:40.686376 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.686036    2552 scope.go:117] "RemoveContainer" containerID="df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87"
Jul 18 01:56:40.695143 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.695126    2552 scope.go:117] "RemoveContainer" containerID="df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87"
Jul 18 01:56:40.695397 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:56:40.695381    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87\": container with ID starting with df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87 not found: ID does not exist" containerID="df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87"
Jul 18 01:56:40.695481 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.695406    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87"} err="failed to get container status \"df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87\": rpc error: code = NotFound desc = could not find container \"df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87\": container with ID starting with df0970362e574383e9c91fc41f5c303060b020e89501bf5974f9c9b72213af87 not found: ID does not exist"
Jul 18 01:56:40.707821 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.707794    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"]
Jul 18 01:56:40.713347 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:40.713323    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kr4qj"]
Jul 18 01:56:41.038139 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.038052    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:41.076012 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.075980    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72d1ee3e-8bc9-4910-8e37-015fb16d5e2d" path="/var/lib/kubelet/pods/72d1ee3e-8bc9-4910-8e37-015fb16d5e2d/volumes"
Jul 18 01:56:41.139407 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.139373    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:41.502601 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.502572    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"]
Jul 18 01:56:41.502922 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.502911    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1f97f22-d7e9-4c27-8c91-c07490fa408c" containerName="authorino"
Jul 18 01:56:41.502966 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.502925    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1f97f22-d7e9-4c27-8c91-c07490fa408c" containerName="authorino"
Jul 18 01:56:41.502966 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.502943    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72d1ee3e-8bc9-4910-8e37-015fb16d5e2d" containerName="limitador"
Jul 18 01:56:41.502966 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.502949    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="72d1ee3e-8bc9-4910-8e37-015fb16d5e2d" containerName="limitador"
Jul 18 01:56:41.503072 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.503021    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1f97f22-d7e9-4c27-8c91-c07490fa408c" containerName="authorino"
Jul 18 01:56:41.503072 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.503031    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="72d1ee3e-8bc9-4910-8e37-015fb16d5e2d" containerName="limitador"
Jul 18 01:56:41.505633 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.505612    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"
Jul 18 01:56:41.508148 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.507972    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 01:56:41.508148 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.507972    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-hgp7c\""
Jul 18 01:56:41.508469 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.508407    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 18 01:56:41.508611 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.508495    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:56:41.517103 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.517078    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"]
Jul 18 01:56:41.540398 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.540351    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:41.672727 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.672692    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62g9f\" (UniqueName: \"kubernetes.io/projected/d8724144-3a41-4f5c-82f2-adb058bac1f6-kube-api-access-62g9f\") pod \"postgres-69fb4594cf-8k7nq\" (UID: \"d8724144-3a41-4f5c-82f2-adb058bac1f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"
Jul 18 01:56:41.672902 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.672733    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d8724144-3a41-4f5c-82f2-adb058bac1f6-data\") pod \"postgres-69fb4594cf-8k7nq\" (UID: \"d8724144-3a41-4f5c-82f2-adb058bac1f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"
Jul 18 01:56:41.773723 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.773631    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62g9f\" (UniqueName: \"kubernetes.io/projected/d8724144-3a41-4f5c-82f2-adb058bac1f6-kube-api-access-62g9f\") pod \"postgres-69fb4594cf-8k7nq\" (UID: \"d8724144-3a41-4f5c-82f2-adb058bac1f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"
Jul 18 01:56:41.773723 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.773682    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d8724144-3a41-4f5c-82f2-adb058bac1f6-data\") pod \"postgres-69fb4594cf-8k7nq\" (UID: \"d8724144-3a41-4f5c-82f2-adb058bac1f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"
Jul 18 01:56:41.774123 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.774091    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d8724144-3a41-4f5c-82f2-adb058bac1f6-data\") pod \"postgres-69fb4594cf-8k7nq\" (UID: \"d8724144-3a41-4f5c-82f2-adb058bac1f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"
Jul 18 01:56:41.783260 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.783237    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62g9f\" (UniqueName: \"kubernetes.io/projected/d8724144-3a41-4f5c-82f2-adb058bac1f6-kube-api-access-62g9f\") pod \"postgres-69fb4594cf-8k7nq\" (UID: \"d8724144-3a41-4f5c-82f2-adb058bac1f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"
Jul 18 01:56:41.819458 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.819410    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"
Jul 18 01:56:41.959760 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:41.959708    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"]
Jul 18 01:56:41.962061 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:56:41.962036    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8724144_3a41_4f5c_82f2_adb058bac1f6.slice/crio-08876d129dfcff0bdf3de6c1ca836a5c040dc2a0e26a18b644598e96f4279f5d WatchSource:0}: Error finding container 08876d129dfcff0bdf3de6c1ca836a5c040dc2a0e26a18b644598e96f4279f5d: Status 404 returned error can't find the container with id 08876d129dfcff0bdf3de6c1ca836a5c040dc2a0e26a18b644598e96f4279f5d
Jul 18 01:56:42.705837 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:42.705802    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq" event={"ID":"d8724144-3a41-4f5c-82f2-adb058bac1f6","Type":"ContainerStarted","Data":"08876d129dfcff0bdf3de6c1ca836a5c040dc2a0e26a18b644598e96f4279f5d"}
Jul 18 01:56:46.732775 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:46.732739    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq" event={"ID":"d8724144-3a41-4f5c-82f2-adb058bac1f6","Type":"ContainerStarted","Data":"a8a418d5ee0c74a5d89654651a6f309da799b70838bf8931ae16ecbef71f66c9"}
Jul 18 01:56:46.733141 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:46.732828    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"
Jul 18 01:56:46.747616 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:46.747552    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq" podStartSLOduration=1.069930281 podStartE2EDuration="5.74753772s" totalImagesPullingTime="4.677607439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:46.746007018 +0000 UTC m=+512.185503970" watchObservedRunningTime="2026-07-18 01:56:46.74753772 +0000 UTC m=+512.187034656"
Jul 18 01:56:52.767160 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:52.767131    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8k7nq"
Jul 18 01:56:53.088452 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.088355    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:53.330287 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.330250    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wkhvz"]
Jul 18 01:56:53.337365 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.337340    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wkhvz"
Jul 18 01:56:53.339711 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.339651    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zklft\""
Jul 18 01:56:53.340819 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.340789    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wkhvz"]
Jul 18 01:56:53.475691 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.475640    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcs2h\" (UniqueName: \"kubernetes.io/projected/1086acfa-06ed-44a6-879b-21d74054b14e-kube-api-access-pcs2h\") pod \"authorino-5d4b67c446-wkhvz\" (UID: \"1086acfa-06ed-44a6-879b-21d74054b14e\") " pod="kuadrant-system/authorino-5d4b67c446-wkhvz"
Jul 18 01:56:53.576234 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.576200    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcs2h\" (UniqueName: \"kubernetes.io/projected/1086acfa-06ed-44a6-879b-21d74054b14e-kube-api-access-pcs2h\") pod \"authorino-5d4b67c446-wkhvz\" (UID: \"1086acfa-06ed-44a6-879b-21d74054b14e\") " pod="kuadrant-system/authorino-5d4b67c446-wkhvz"
Jul 18 01:56:53.585163 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.585135    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcs2h\" (UniqueName: \"kubernetes.io/projected/1086acfa-06ed-44a6-879b-21d74054b14e-kube-api-access-pcs2h\") pod \"authorino-5d4b67c446-wkhvz\" (UID: \"1086acfa-06ed-44a6-879b-21d74054b14e\") " pod="kuadrant-system/authorino-5d4b67c446-wkhvz"
Jul 18 01:56:53.590169 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.590110    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wkhvz"]
Jul 18 01:56:53.590399 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.590386    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wkhvz"
Jul 18 01:56:53.597653 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.597626    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:53.617081 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.617043    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6476d6d58f-2h78b"]
Jul 18 01:56:53.622384 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.622350    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6476d6d58f-2h78b"
Jul 18 01:56:53.625764 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.625743    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6476d6d58f-2h78b"]
Jul 18 01:56:53.729971 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.729946    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wkhvz"]
Jul 18 01:56:53.732137 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:56:53.732106    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1086acfa_06ed_44a6_879b_21d74054b14e.slice/crio-d3eaf9916b697850ffffc131aeb217ce8da64c13c64c5d17df9f12b8d879ec12 WatchSource:0}: Error finding container d3eaf9916b697850ffffc131aeb217ce8da64c13c64c5d17df9f12b8d879ec12: Status 404 returned error can't find the container with id d3eaf9916b697850ffffc131aeb217ce8da64c13c64c5d17df9f12b8d879ec12
Jul 18 01:56:53.771573 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.771546    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wkhvz" event={"ID":"1086acfa-06ed-44a6-879b-21d74054b14e","Type":"ContainerStarted","Data":"d3eaf9916b697850ffffc131aeb217ce8da64c13c64c5d17df9f12b8d879ec12"}
Jul 18 01:56:53.778978 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.778952    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26sbg\" (UniqueName: \"kubernetes.io/projected/073e1202-5616-4008-b8bc-51f60b187d27-kube-api-access-26sbg\") pod \"authorino-6476d6d58f-2h78b\" (UID: \"073e1202-5616-4008-b8bc-51f60b187d27\") " pod="kuadrant-system/authorino-6476d6d58f-2h78b"
Jul 18 01:56:53.879410 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.879329    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6476d6d58f-2h78b"]
Jul 18 01:56:53.879605 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:56:53.879589    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-26sbg], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6476d6d58f-2h78b" podUID="073e1202-5616-4008-b8bc-51f60b187d27"
Jul 18 01:56:53.879666 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.879648    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26sbg\" (UniqueName: \"kubernetes.io/projected/073e1202-5616-4008-b8bc-51f60b187d27-kube-api-access-26sbg\") pod \"authorino-6476d6d58f-2h78b\" (UID: \"073e1202-5616-4008-b8bc-51f60b187d27\") " pod="kuadrant-system/authorino-6476d6d58f-2h78b"
Jul 18 01:56:53.894005 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.893943    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26sbg\" (UniqueName: \"kubernetes.io/projected/073e1202-5616-4008-b8bc-51f60b187d27-kube-api-access-26sbg\") pod \"authorino-6476d6d58f-2h78b\" (UID: \"073e1202-5616-4008-b8bc-51f60b187d27\") " pod="kuadrant-system/authorino-6476d6d58f-2h78b"
Jul 18 01:56:53.905503 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.905479    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5b7456c76f-bxb5v"]
Jul 18 01:56:53.911057 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.911037    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b7456c76f-bxb5v"
Jul 18 01:56:53.913437 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.913400    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 18 01:56:53.916033 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:53.916011    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b7456c76f-bxb5v"]
Jul 18 01:56:54.081470 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.081417    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgvr2\" (UniqueName: \"kubernetes.io/projected/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-kube-api-access-kgvr2\") pod \"authorino-5b7456c76f-bxb5v\" (UID: \"a747c6ed-93e0-41e4-8e24-b4c660de5b7b\") " pod="kuadrant-system/authorino-5b7456c76f-bxb5v"
Jul 18 01:56:54.081626 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.081494    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-tls-cert\") pod \"authorino-5b7456c76f-bxb5v\" (UID: \"a747c6ed-93e0-41e4-8e24-b4c660de5b7b\") " pod="kuadrant-system/authorino-5b7456c76f-bxb5v"
Jul 18 01:56:54.182170 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.182140    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgvr2\" (UniqueName: \"kubernetes.io/projected/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-kube-api-access-kgvr2\") pod \"authorino-5b7456c76f-bxb5v\" (UID: \"a747c6ed-93e0-41e4-8e24-b4c660de5b7b\") " pod="kuadrant-system/authorino-5b7456c76f-bxb5v"
Jul 18 01:56:54.182321 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.182189    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-tls-cert\") pod \"authorino-5b7456c76f-bxb5v\" (UID: \"a747c6ed-93e0-41e4-8e24-b4c660de5b7b\") " pod="kuadrant-system/authorino-5b7456c76f-bxb5v"
Jul 18 01:56:54.185095 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.185068    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-tls-cert\") pod \"authorino-5b7456c76f-bxb5v\" (UID: \"a747c6ed-93e0-41e4-8e24-b4c660de5b7b\") " pod="kuadrant-system/authorino-5b7456c76f-bxb5v"
Jul 18 01:56:54.189849 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.189826    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgvr2\" (UniqueName: \"kubernetes.io/projected/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-kube-api-access-kgvr2\") pod \"authorino-5b7456c76f-bxb5v\" (UID: \"a747c6ed-93e0-41e4-8e24-b4c660de5b7b\") " pod="kuadrant-system/authorino-5b7456c76f-bxb5v"
Jul 18 01:56:54.221817 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.221776    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b7456c76f-bxb5v"
Jul 18 01:56:54.359302 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.359277    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b7456c76f-bxb5v"]
Jul 18 01:56:54.360891 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:56:54.360856    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda747c6ed_93e0_41e4_8e24_b4c660de5b7b.slice/crio-e64794254c3e4bf76c1dba57efb9612e53ce6335af9c7726d8bec7932155c08c WatchSource:0}: Error finding container e64794254c3e4bf76c1dba57efb9612e53ce6335af9c7726d8bec7932155c08c: Status 404 returned error can't find the container with id e64794254c3e4bf76c1dba57efb9612e53ce6335af9c7726d8bec7932155c08c
Jul 18 01:56:54.779500 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.779462    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wkhvz" event={"ID":"1086acfa-06ed-44a6-879b-21d74054b14e","Type":"ContainerStarted","Data":"c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de"}
Jul 18 01:56:54.779957 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.779531    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-wkhvz" podUID="1086acfa-06ed-44a6-879b-21d74054b14e" containerName="authorino" containerID="cri-o://c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de" gracePeriod=30
Jul 18 01:56:54.781159 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.781132    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b7456c76f-bxb5v" event={"ID":"a747c6ed-93e0-41e4-8e24-b4c660de5b7b","Type":"ContainerStarted","Data":"f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8"}
Jul 18 01:56:54.781288 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.781168    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b7456c76f-bxb5v" event={"ID":"a747c6ed-93e0-41e4-8e24-b4c660de5b7b","Type":"ContainerStarted","Data":"e64794254c3e4bf76c1dba57efb9612e53ce6335af9c7726d8bec7932155c08c"}
Jul 18 01:56:54.781288 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.781204    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6476d6d58f-2h78b"
Jul 18 01:56:54.790012 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.789911    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6476d6d58f-2h78b"
Jul 18 01:56:54.796983 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.796935    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-wkhvz" podStartSLOduration=1.43897873 podStartE2EDuration="1.796917482s" totalImagesPullingTime="357.938752ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:54.794784934 +0000 UTC m=+520.234281867" watchObservedRunningTime="2026-07-18 01:56:54.796917482 +0000 UTC m=+520.236414416"
Jul 18 01:56:54.888250 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.888221    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26sbg\" (UniqueName: \"kubernetes.io/projected/073e1202-5616-4008-b8bc-51f60b187d27-kube-api-access-26sbg\") pod \"073e1202-5616-4008-b8bc-51f60b187d27\" (UID: \"073e1202-5616-4008-b8bc-51f60b187d27\") "
Jul 18 01:56:54.890407 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.890373    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/073e1202-5616-4008-b8bc-51f60b187d27-kube-api-access-26sbg" (OuterVolumeSpecName: "kube-api-access-26sbg") pod "073e1202-5616-4008-b8bc-51f60b187d27" (UID: "073e1202-5616-4008-b8bc-51f60b187d27"). InnerVolumeSpecName "kube-api-access-26sbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:54.988999 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:54.988971    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26sbg\" (UniqueName: \"kubernetes.io/projected/073e1202-5616-4008-b8bc-51f60b187d27-kube-api-access-26sbg\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:56:55.028716 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.028691    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wkhvz"
Jul 18 01:56:55.047774 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.046217    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5b7456c76f-bxb5v" podStartSLOduration=1.714574812 podStartE2EDuration="2.046080943s" totalImagesPullingTime="331.506131ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:54.826695198 +0000 UTC m=+520.266192132" watchObservedRunningTime="2026-07-18 01:56:55.046080943 +0000 UTC m=+520.485577878"
Jul 18 01:56:55.190918 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.190878    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcs2h\" (UniqueName: \"kubernetes.io/projected/1086acfa-06ed-44a6-879b-21d74054b14e-kube-api-access-pcs2h\") pod \"1086acfa-06ed-44a6-879b-21d74054b14e\" (UID: \"1086acfa-06ed-44a6-879b-21d74054b14e\") "
Jul 18 01:56:55.193151 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.193125    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1086acfa-06ed-44a6-879b-21d74054b14e-kube-api-access-pcs2h" (OuterVolumeSpecName: "kube-api-access-pcs2h") pod "1086acfa-06ed-44a6-879b-21d74054b14e" (UID: "1086acfa-06ed-44a6-879b-21d74054b14e"). InnerVolumeSpecName "kube-api-access-pcs2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:55.292394 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.292317    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcs2h\" (UniqueName: \"kubernetes.io/projected/1086acfa-06ed-44a6-879b-21d74054b14e-kube-api-access-pcs2h\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:56:55.788691 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.788658    2552 generic.go:356] "Generic (PLEG): container finished" podID="1086acfa-06ed-44a6-879b-21d74054b14e" containerID="c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de" exitCode=0
Jul 18 01:56:55.789159 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.788724    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wkhvz" event={"ID":"1086acfa-06ed-44a6-879b-21d74054b14e","Type":"ContainerDied","Data":"c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de"}
Jul 18 01:56:55.789159 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.788746    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wkhvz"
Jul 18 01:56:55.789159 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.788778    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wkhvz" event={"ID":"1086acfa-06ed-44a6-879b-21d74054b14e","Type":"ContainerDied","Data":"d3eaf9916b697850ffffc131aeb217ce8da64c13c64c5d17df9f12b8d879ec12"}
Jul 18 01:56:55.789159 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.788800    2552 scope.go:117] "RemoveContainer" containerID="c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de"
Jul 18 01:56:55.789159 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.788999    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6476d6d58f-2h78b"
Jul 18 01:56:55.799963 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.799944    2552 scope.go:117] "RemoveContainer" containerID="c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de"
Jul 18 01:56:55.800274 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:56:55.800255    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de\": container with ID starting with c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de not found: ID does not exist" containerID="c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de"
Jul 18 01:56:55.800327 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.800283    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de"} err="failed to get container status \"c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de\": rpc error: code = NotFound desc = could not find container \"c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de\": container with ID starting with c194f24d1c25457d7d569ec4e5c608bf3de10da74d29199542cf7ccfa5f4c5de not found: ID does not exist"
Jul 18 01:56:55.818944 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.818913    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6476d6d58f-2h78b"]
Jul 18 01:56:55.824274 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.824248    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6476d6d58f-2h78b"]
Jul 18 01:56:55.834075 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.834046    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wkhvz"]
Jul 18 01:56:55.838280 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:55.838259    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wkhvz"]
Jul 18 01:56:57.076680 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:57.076648    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="073e1202-5616-4008-b8bc-51f60b187d27" path="/var/lib/kubelet/pods/073e1202-5616-4008-b8bc-51f60b187d27/volumes"
Jul 18 01:56:57.077099 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:57.076939    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1086acfa-06ed-44a6-879b-21d74054b14e" path="/var/lib/kubelet/pods/1086acfa-06ed-44a6-879b-21d74054b14e/volumes"
Jul 18 01:56:59.845663 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:59.845621    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:56:59.849811 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:59.849780    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-758d9f5f8c-njg6n"]
Jul 18 01:56:59.850160 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:59.850149    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1086acfa-06ed-44a6-879b-21d74054b14e" containerName="authorino"
Jul 18 01:56:59.850203 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:59.850163    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1086acfa-06ed-44a6-879b-21d74054b14e" containerName="authorino"
Jul 18 01:56:59.850238 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:59.850228    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1086acfa-06ed-44a6-879b-21d74054b14e" containerName="authorino"
Jul 18 01:56:59.943341 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:56:59.943303    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:00.009548 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.009520    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-758d9f5f8c-njg6n"]
Jul 18 01:57:00.009717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.009646    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:00.012126 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.012106    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wb6dd\""
Jul 18 01:57:00.012306 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.012150    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 18 01:57:00.012641 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.012624    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 18 01:57:00.030176 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.030151    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rk2m\" (UniqueName: \"kubernetes.io/projected/a7af76e3-2d6f-452e-9043-207d6efef53a-kube-api-access-4rk2m\") pod \"maas-controller-758d9f5f8c-njg6n\" (UID: \"a7af76e3-2d6f-452e-9043-207d6efef53a\") " pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:00.030299 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.030210    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7af76e3-2d6f-452e-9043-207d6efef53a-webhook-cert\") pod \"maas-controller-758d9f5f8c-njg6n\" (UID: \"a7af76e3-2d6f-452e-9043-207d6efef53a\") " pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:00.035929 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.035899    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:00.040924 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.040903    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d774b7794-trxv6"]
Jul 18 01:57:00.054881 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.054859    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 01:57:00.056694 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.056673    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d774b7794-trxv6"]
Jul 18 01:57:00.104487 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.104398    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:00.131256 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.131229    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3076ffb6-79e5-4d5c-9764-e23e50bad43a-webhook-cert\") pod \"maas-controller-5d774b7794-trxv6\" (UID: \"3076ffb6-79e5-4d5c-9764-e23e50bad43a\") " pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 01:57:00.131382 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.131268    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lrm9\" (UniqueName: \"kubernetes.io/projected/3076ffb6-79e5-4d5c-9764-e23e50bad43a-kube-api-access-5lrm9\") pod \"maas-controller-5d774b7794-trxv6\" (UID: \"3076ffb6-79e5-4d5c-9764-e23e50bad43a\") " pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 01:57:00.131382 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.131326    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7af76e3-2d6f-452e-9043-207d6efef53a-webhook-cert\") pod \"maas-controller-758d9f5f8c-njg6n\" (UID: \"a7af76e3-2d6f-452e-9043-207d6efef53a\") " pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:00.131492 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.131408    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rk2m\" (UniqueName: \"kubernetes.io/projected/a7af76e3-2d6f-452e-9043-207d6efef53a-kube-api-access-4rk2m\") pod \"maas-controller-758d9f5f8c-njg6n\" (UID: \"a7af76e3-2d6f-452e-9043-207d6efef53a\") " pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:00.133932 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.133914    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7af76e3-2d6f-452e-9043-207d6efef53a-webhook-cert\") pod \"maas-controller-758d9f5f8c-njg6n\" (UID: \"a7af76e3-2d6f-452e-9043-207d6efef53a\") " pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:00.139933 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.139913    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rk2m\" (UniqueName: \"kubernetes.io/projected/a7af76e3-2d6f-452e-9043-207d6efef53a-kube-api-access-4rk2m\") pod \"maas-controller-758d9f5f8c-njg6n\" (UID: \"a7af76e3-2d6f-452e-9043-207d6efef53a\") " pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:00.232075 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.232043    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3076ffb6-79e5-4d5c-9764-e23e50bad43a-webhook-cert\") pod \"maas-controller-5d774b7794-trxv6\" (UID: \"3076ffb6-79e5-4d5c-9764-e23e50bad43a\") " pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 01:57:00.232261 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.232086    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lrm9\" (UniqueName: \"kubernetes.io/projected/3076ffb6-79e5-4d5c-9764-e23e50bad43a-kube-api-access-5lrm9\") pod \"maas-controller-5d774b7794-trxv6\" (UID: \"3076ffb6-79e5-4d5c-9764-e23e50bad43a\") " pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 01:57:00.234665 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.234643    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3076ffb6-79e5-4d5c-9764-e23e50bad43a-webhook-cert\") pod \"maas-controller-5d774b7794-trxv6\" (UID: \"3076ffb6-79e5-4d5c-9764-e23e50bad43a\") " pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 01:57:00.243677 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.243649    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lrm9\" (UniqueName: \"kubernetes.io/projected/3076ffb6-79e5-4d5c-9764-e23e50bad43a-kube-api-access-5lrm9\") pod \"maas-controller-5d774b7794-trxv6\" (UID: \"3076ffb6-79e5-4d5c-9764-e23e50bad43a\") " pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 01:57:00.320503 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.320468    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:00.365878 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.365797    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 01:57:00.487133 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.487068    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-758d9f5f8c-njg6n"]
Jul 18 01:57:00.488762 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:57:00.488708    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7af76e3_2d6f_452e_9043_207d6efef53a.slice/crio-4398ade74df70535ca8362e52cfa640934128e8442ca9f5e596efa3380244bde WatchSource:0}: Error finding container 4398ade74df70535ca8362e52cfa640934128e8442ca9f5e596efa3380244bde: Status 404 returned error can't find the container with id 4398ade74df70535ca8362e52cfa640934128e8442ca9f5e596efa3380244bde
Jul 18 01:57:00.539332 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.539308    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d774b7794-trxv6"]
Jul 18 01:57:00.541342 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:57:00.541311    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3076ffb6_79e5_4d5c_9764_e23e50bad43a.slice/crio-29214e7cb11cf30175c70702f03c0b395f9e04938979c2992e2580a0e51a2816 WatchSource:0}: Error finding container 29214e7cb11cf30175c70702f03c0b395f9e04938979c2992e2580a0e51a2816: Status 404 returned error can't find the container with id 29214e7cb11cf30175c70702f03c0b395f9e04938979c2992e2580a0e51a2816
Jul 18 01:57:00.817852 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.817819    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d774b7794-trxv6" event={"ID":"3076ffb6-79e5-4d5c-9764-e23e50bad43a","Type":"ContainerStarted","Data":"29214e7cb11cf30175c70702f03c0b395f9e04938979c2992e2580a0e51a2816"}
Jul 18 01:57:00.819094 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:00.819069    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758d9f5f8c-njg6n" event={"ID":"a7af76e3-2d6f-452e-9043-207d6efef53a","Type":"ContainerStarted","Data":"4398ade74df70535ca8362e52cfa640934128e8442ca9f5e596efa3380244bde"}
Jul 18 01:57:03.397060 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:03.397024    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:03.493725 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:03.493639    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:03.842842 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:03.842731    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758d9f5f8c-njg6n" event={"ID":"a7af76e3-2d6f-452e-9043-207d6efef53a","Type":"ContainerStarted","Data":"211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7"}
Jul 18 01:57:03.842842 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:03.842810    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:03.844562 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:03.844536    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d774b7794-trxv6" event={"ID":"3076ffb6-79e5-4d5c-9764-e23e50bad43a","Type":"ContainerStarted","Data":"4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea"}
Jul 18 01:57:03.844711 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:03.844663    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 01:57:03.871193 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:03.871129    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-758d9f5f8c-njg6n" podStartSLOduration=2.191857487 podStartE2EDuration="4.871108044s" totalImagesPullingTime="2.679250557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:03.868759688 +0000 UTC m=+529.308256637" watchObservedRunningTime="2026-07-18 01:57:03.871108044 +0000 UTC m=+529.310604977"
Jul 18 01:57:03.893044 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:03.892997    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5d774b7794-trxv6" podStartSLOduration=1.255465978 podStartE2EDuration="3.892980026s" totalImagesPullingTime="2.637514048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:03.89207949 +0000 UTC m=+529.331576413" watchObservedRunningTime="2026-07-18 01:57:03.892980026 +0000 UTC m=+529.332476958"
Jul 18 01:57:14.852476 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:14.852446    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:14.852971 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:14.852533    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 01:57:14.912434 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:14.912400    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-758d9f5f8c-njg6n"]
Jul 18 01:57:14.912634 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:14.912610    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-758d9f5f8c-njg6n" podUID="a7af76e3-2d6f-452e-9043-207d6efef53a" containerName="manager" containerID="cri-o://211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7" gracePeriod=30
Jul 18 01:57:15.172310 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.172281    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:15.258179 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.258150    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rk2m\" (UniqueName: \"kubernetes.io/projected/a7af76e3-2d6f-452e-9043-207d6efef53a-kube-api-access-4rk2m\") pod \"a7af76e3-2d6f-452e-9043-207d6efef53a\" (UID: \"a7af76e3-2d6f-452e-9043-207d6efef53a\") "
Jul 18 01:57:15.258375 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.258186    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7af76e3-2d6f-452e-9043-207d6efef53a-webhook-cert\") pod \"a7af76e3-2d6f-452e-9043-207d6efef53a\" (UID: \"a7af76e3-2d6f-452e-9043-207d6efef53a\") "
Jul 18 01:57:15.260628 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.260595    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7af76e3-2d6f-452e-9043-207d6efef53a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a7af76e3-2d6f-452e-9043-207d6efef53a" (UID: "a7af76e3-2d6f-452e-9043-207d6efef53a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:57:15.260628 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.260619    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7af76e3-2d6f-452e-9043-207d6efef53a-kube-api-access-4rk2m" (OuterVolumeSpecName: "kube-api-access-4rk2m") pod "a7af76e3-2d6f-452e-9043-207d6efef53a" (UID: "a7af76e3-2d6f-452e-9043-207d6efef53a"). InnerVolumeSpecName "kube-api-access-4rk2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:57:15.359113 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.359056    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rk2m\" (UniqueName: \"kubernetes.io/projected/a7af76e3-2d6f-452e-9043-207d6efef53a-kube-api-access-4rk2m\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:57:15.359113 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.359104    2552 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7af76e3-2d6f-452e-9043-207d6efef53a-webhook-cert\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 01:57:15.908227 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.908187    2552 generic.go:356] "Generic (PLEG): container finished" podID="a7af76e3-2d6f-452e-9043-207d6efef53a" containerID="211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7" exitCode=0
Jul 18 01:57:15.908747 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.908235    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758d9f5f8c-njg6n"
Jul 18 01:57:15.908747 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.908261    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758d9f5f8c-njg6n" event={"ID":"a7af76e3-2d6f-452e-9043-207d6efef53a","Type":"ContainerDied","Data":"211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7"}
Jul 18 01:57:15.908747 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.908307    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758d9f5f8c-njg6n" event={"ID":"a7af76e3-2d6f-452e-9043-207d6efef53a","Type":"ContainerDied","Data":"4398ade74df70535ca8362e52cfa640934128e8442ca9f5e596efa3380244bde"}
Jul 18 01:57:15.908747 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.908331    2552 scope.go:117] "RemoveContainer" containerID="211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7"
Jul 18 01:57:15.917972 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.917826    2552 scope.go:117] "RemoveContainer" containerID="211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7"
Jul 18 01:57:15.918094 ip-10-0-128-48 kubenswrapper[2552]: E0718 01:57:15.918076    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7\": container with ID starting with 211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7 not found: ID does not exist" containerID="211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7"
Jul 18 01:57:15.918131 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.918103    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7"} err="failed to get container status \"211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7\": rpc error: code = NotFound desc = could not find container \"211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7\": container with ID starting with 211c51b0932414eb9676a16d1788639ea600e9c560aa4e998bceb625b7c5c1a7 not found: ID does not exist"
Jul 18 01:57:15.928497 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.928472    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-758d9f5f8c-njg6n"]
Jul 18 01:57:15.932227 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:15.932200    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-758d9f5f8c-njg6n"]
Jul 18 01:57:17.076231 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:17.076204    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7af76e3-2d6f-452e-9043-207d6efef53a" path="/var/lib/kubelet/pods/a7af76e3-2d6f-452e-9043-207d6efef53a/volumes"
Jul 18 01:57:36.821061 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:36.821014    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:36.918046 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:36.918011    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:37.353303 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.353265    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:37.461158 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.461125    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"]
Jul 18 01:57:37.461541 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.461526    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7af76e3-2d6f-452e-9043-207d6efef53a" containerName="manager"
Jul 18 01:57:37.461541 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.461542    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7af76e3-2d6f-452e-9043-207d6efef53a" containerName="manager"
Jul 18 01:57:37.461676 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.461618    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7af76e3-2d6f-452e-9043-207d6efef53a" containerName="manager"
Jul 18 01:57:37.467335 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.467320    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:37.469548 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.469531    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4q4w8\""
Jul 18 01:57:37.469954 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.469939    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 18 01:57:37.478214 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.478192    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"]
Jul 18 01:57:37.532335 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.532300    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:37.550542 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.550510    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1-maas-api-tls\") pod \"maas-api-6447d79b77-98lrf\" (UID: \"79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1\") " pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:37.550707 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.550582    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1-openshift-service-ca\") pod \"maas-api-6447d79b77-98lrf\" (UID: \"79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1\") " pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:37.550707 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.550668    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqfn8\" (UniqueName: \"kubernetes.io/projected/79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1-kube-api-access-bqfn8\") pod \"maas-api-6447d79b77-98lrf\" (UID: \"79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1\") " pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:37.651121 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.651090    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1-openshift-service-ca\") pod \"maas-api-6447d79b77-98lrf\" (UID: \"79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1\") " pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:37.651290 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.651134    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqfn8\" (UniqueName: \"kubernetes.io/projected/79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1-kube-api-access-bqfn8\") pod \"maas-api-6447d79b77-98lrf\" (UID: \"79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1\") " pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:37.651290 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.651169    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1-maas-api-tls\") pod \"maas-api-6447d79b77-98lrf\" (UID: \"79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1\") " pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:37.651793 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.651770    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1-openshift-service-ca\") pod \"maas-api-6447d79b77-98lrf\" (UID: \"79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1\") " pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:37.653733 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.653714    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1-maas-api-tls\") pod \"maas-api-6447d79b77-98lrf\" (UID: \"79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1\") " pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:37.659739 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.659713    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqfn8\" (UniqueName: \"kubernetes.io/projected/79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1-kube-api-access-bqfn8\") pod \"maas-api-6447d79b77-98lrf\" (UID: \"79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1\") " pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:37.778515 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.778485    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:37.894721 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.894626    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:37.930468 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.930446    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"]
Jul 18 01:57:37.932823 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:57:37.932796    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79dd6c14_c2f5_4d6a_9188_f7e3530ee9b1.slice/crio-cbea90213ab4efd7d88dd4d913ebf4e3621e5d713b32d3a5a388d087df8e202f WatchSource:0}: Error finding container cbea90213ab4efd7d88dd4d913ebf4e3621e5d713b32d3a5a388d087df8e202f: Status 404 returned error can't find the container with id cbea90213ab4efd7d88dd4d913ebf4e3621e5d713b32d3a5a388d087df8e202f
Jul 18 01:57:37.990067 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.990038    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-rjql6"]
Jul 18 01:57:37.992269 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.992249    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:37.993496 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.993481    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6"
Jul 18 01:57:37.995661 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:37.995636    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 18 01:57:38.006064 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.006040    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-rjql6"]
Jul 18 01:57:38.025579 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.025549    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf" event={"ID":"79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1","Type":"ContainerStarted","Data":"cbea90213ab4efd7d88dd4d913ebf4e3621e5d713b32d3a5a388d087df8e202f"}
Jul 18 01:57:38.031288 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.031265    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-5lgnb"]
Jul 18 01:57:38.033649 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.033635    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb"
Jul 18 01:57:38.036456 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.036435    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-kpkm5\""
Jul 18 01:57:38.053363 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.050368    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-5lgnb"]
Jul 18 01:57:38.055561 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.055481    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea035ab2-c88a-4a1b-bc9b-826b51ee9474-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-rjql6\" (UID: \"ea035ab2-c88a-4a1b-bc9b-826b51ee9474\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6"
Jul 18 01:57:38.055561 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.055543    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcdkx\" (UniqueName: \"kubernetes.io/projected/ea035ab2-c88a-4a1b-bc9b-826b51ee9474-kube-api-access-pcdkx\") pod \"payload-pre-processing-7677b774bb-rjql6\" (UID: \"ea035ab2-c88a-4a1b-bc9b-826b51ee9474\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6"
Jul 18 01:57:38.156350 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.156315    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea035ab2-c88a-4a1b-bc9b-826b51ee9474-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-rjql6\" (UID: \"ea035ab2-c88a-4a1b-bc9b-826b51ee9474\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6"
Jul 18 01:57:38.156528 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.156362    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcdkx\" (UniqueName: \"kubernetes.io/projected/ea035ab2-c88a-4a1b-bc9b-826b51ee9474-kube-api-access-pcdkx\") pod \"payload-pre-processing-7677b774bb-rjql6\" (UID: \"ea035ab2-c88a-4a1b-bc9b-826b51ee9474\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6"
Jul 18 01:57:38.156528 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.156400    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb3ee220-0029-4f10-baef-0a67a9893fd8-plugins-config-volume\") pod \"payload-processing-5bf47cd988-5lgnb\" (UID: \"fb3ee220-0029-4f10-baef-0a67a9893fd8\") " pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb"
Jul 18 01:57:38.156528 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.156443    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl7rj\" (UniqueName: \"kubernetes.io/projected/fb3ee220-0029-4f10-baef-0a67a9893fd8-kube-api-access-fl7rj\") pod \"payload-processing-5bf47cd988-5lgnb\" (UID: \"fb3ee220-0029-4f10-baef-0a67a9893fd8\") " pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb"
Jul 18 01:57:38.157021 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.156999    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea035ab2-c88a-4a1b-bc9b-826b51ee9474-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-rjql6\" (UID: \"ea035ab2-c88a-4a1b-bc9b-826b51ee9474\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6"
Jul 18 01:57:38.164150 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.164117    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcdkx\" (UniqueName: \"kubernetes.io/projected/ea035ab2-c88a-4a1b-bc9b-826b51ee9474-kube-api-access-pcdkx\") pod \"payload-pre-processing-7677b774bb-rjql6\" (UID: \"ea035ab2-c88a-4a1b-bc9b-826b51ee9474\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6"
Jul 18 01:57:38.257972 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.257887    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl7rj\" (UniqueName: \"kubernetes.io/projected/fb3ee220-0029-4f10-baef-0a67a9893fd8-kube-api-access-fl7rj\") pod \"payload-processing-5bf47cd988-5lgnb\" (UID: \"fb3ee220-0029-4f10-baef-0a67a9893fd8\") " pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb"
Jul 18 01:57:38.258124 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.258013    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb3ee220-0029-4f10-baef-0a67a9893fd8-plugins-config-volume\") pod \"payload-processing-5bf47cd988-5lgnb\" (UID: \"fb3ee220-0029-4f10-baef-0a67a9893fd8\") " pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb"
Jul 18 01:57:38.258905 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.258879    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb3ee220-0029-4f10-baef-0a67a9893fd8-plugins-config-volume\") pod \"payload-processing-5bf47cd988-5lgnb\" (UID: \"fb3ee220-0029-4f10-baef-0a67a9893fd8\") " pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb"
Jul 18 01:57:38.265783 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.265761    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl7rj\" (UniqueName: \"kubernetes.io/projected/fb3ee220-0029-4f10-baef-0a67a9893fd8-kube-api-access-fl7rj\") pod \"payload-processing-5bf47cd988-5lgnb\" (UID: \"fb3ee220-0029-4f10-baef-0a67a9893fd8\") " pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb"
Jul 18 01:57:38.304960 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.304931    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6"
Jul 18 01:57:38.343546 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.343516    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb"
Jul 18 01:57:38.463662 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.463627    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-rjql6"]
Jul 18 01:57:38.464847 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:57:38.464821    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea035ab2_c88a_4a1b_bc9b_826b51ee9474.slice/crio-b3cddbcff8b3a18b3d851dd3472a579e75022eb15a734c1a77050185b3c14dce WatchSource:0}: Error finding container b3cddbcff8b3a18b3d851dd3472a579e75022eb15a734c1a77050185b3c14dce: Status 404 returned error can't find the container with id b3cddbcff8b3a18b3d851dd3472a579e75022eb15a734c1a77050185b3c14dce
Jul 18 01:57:38.502637 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:38.502571    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-5lgnb"]
Jul 18 01:57:38.506989 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:57:38.506956    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb3ee220_0029_4f10_baef_0a67a9893fd8.slice/crio-ccfedc8ac89ea9bcb1467ed9c99c3482ae7e5e0f37792abfe2daa240ada55060 WatchSource:0}: Error finding container ccfedc8ac89ea9bcb1467ed9c99c3482ae7e5e0f37792abfe2daa240ada55060: Status 404 returned error can't find the container with id ccfedc8ac89ea9bcb1467ed9c99c3482ae7e5e0f37792abfe2daa240ada55060
Jul 18 01:57:39.038274 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:39.038188    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb" event={"ID":"fb3ee220-0029-4f10-baef-0a67a9893fd8","Type":"ContainerStarted","Data":"ccfedc8ac89ea9bcb1467ed9c99c3482ae7e5e0f37792abfe2daa240ada55060"}
Jul 18 01:57:39.041301 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:39.041231    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6" event={"ID":"ea035ab2-c88a-4a1b-bc9b-826b51ee9474","Type":"ContainerStarted","Data":"b3cddbcff8b3a18b3d851dd3472a579e75022eb15a734c1a77050185b3c14dce"}
Jul 18 01:57:40.049277 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:40.049191    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf" event={"ID":"79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1","Type":"ContainerStarted","Data":"d6dc0a2a8857e11be8c8080036a39e150c0d0d9e459e153c998369f3ca8fc81f"}
Jul 18 01:57:40.049743 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:40.049343    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:40.067531 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:40.067474    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf" podStartSLOduration=1.317253945 podStartE2EDuration="3.067457993s" totalImagesPullingTime="1.750204048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:40.065884985 +0000 UTC m=+565.505381918" watchObservedRunningTime="2026-07-18 01:57:40.067457993 +0000 UTC m=+565.506954949"
Jul 18 01:57:41.059236 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:41.059196    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb" event={"ID":"fb3ee220-0029-4f10-baef-0a67a9893fd8","Type":"ContainerStarted","Data":"c4e3ca0f13fca6b45be3dcdd4e538d5fe9be04115077a179af077d9543e14acf"}
Jul 18 01:57:41.059750 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:41.059314    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb"
Jul 18 01:57:41.061020 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:41.060995    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6" event={"ID":"ea035ab2-c88a-4a1b-bc9b-826b51ee9474","Type":"ContainerStarted","Data":"4c8f24b6ae46733fe2a4a2f70c47fc161c56edba8417e37b8fa6c697e4839bc4"}
Jul 18 01:57:41.061154 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:41.061097    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6"
Jul 18 01:57:41.074921 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:41.074869    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb" podStartSLOduration=1.347742153 podStartE2EDuration="3.074852316s" totalImagesPullingTime="1.727110163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:41.073068966 +0000 UTC m=+566.512565908" watchObservedRunningTime="2026-07-18 01:57:41.074852316 +0000 UTC m=+566.514349247"
Jul 18 01:57:41.086479 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:41.086412    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6" podStartSLOduration=2.3227137239999998 podStartE2EDuration="4.086398155s" totalImagesPullingTime="1.763684431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:41.086132728 +0000 UTC m=+566.525629660" watchObservedRunningTime="2026-07-18 01:57:41.086398155 +0000 UTC m=+566.525895087"
Jul 18 01:57:46.067592 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:46.067564    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6447d79b77-98lrf"
Jul 18 01:57:47.089800 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:47.089766    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:47.186566 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:47.186525    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:57:52.069615 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:52.069583    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-5lgnb"
Jul 18 01:57:52.070010 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:57:52.069634    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-rjql6"
Jul 18 01:58:04.302003 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:04.301964    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:04.357846 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:04.357815    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:08.285099 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:08.285033    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:08.345525 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:08.345494    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:10.007725 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:10.007693    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:14.984481 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:14.984449    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 01:58:14.985033 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:14.985015    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 01:58:19.167827 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.167787    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"]
Jul 18 01:58:19.171287 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.171272    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.173882 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.173858    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 18 01:58:19.174024 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.173902    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tfrql\""
Jul 18 01:58:19.174024 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.173906    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 18 01:58:19.174024 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.173963    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 18 01:58:19.180657 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.180630    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"]
Jul 18 01:58:19.201437 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.201395    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.201562 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.201466    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65f40363-9b2c-483a-8d8e-5fad3f554e3b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.201562 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.201504    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cczsl\" (UniqueName: \"kubernetes.io/projected/65f40363-9b2c-483a-8d8e-5fad3f554e3b-kube-api-access-cczsl\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.201562 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.201527    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.201562 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.201545    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.201726 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.201657    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.261351 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.261299    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:19.302445 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.302395    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.302625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.302463    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.302625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.302550    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.302625 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.302582    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.302754 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.302731    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65f40363-9b2c-483a-8d8e-5fad3f554e3b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.302808 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.302781    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cczsl\" (UniqueName: \"kubernetes.io/projected/65f40363-9b2c-483a-8d8e-5fad3f554e3b-kube-api-access-cczsl\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.302919 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.302900    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.302978 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.302947    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.303016 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.302989    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.304950 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.304928    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65f40363-9b2c-483a-8d8e-5fad3f554e3b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.305203 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.305187    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65f40363-9b2c-483a-8d8e-5fad3f554e3b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.311568 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.311540    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cczsl\" (UniqueName: \"kubernetes.io/projected/65f40363-9b2c-483a-8d8e-5fad3f554e3b-kube-api-access-cczsl\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c889s\" (UID: \"65f40363-9b2c-483a-8d8e-5fad3f554e3b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.482644 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.482558    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:19.634636 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.632077    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"]
Jul 18 01:58:19.636446 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:58:19.636396    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f40363_9b2c_483a_8d8e_5fad3f554e3b.slice/crio-52ea801004962c05657818d45b2058cbfb2f3fb861d7a96720a0a6b3da851cbb WatchSource:0}: Error finding container 52ea801004962c05657818d45b2058cbfb2f3fb861d7a96720a0a6b3da851cbb: Status 404 returned error can't find the container with id 52ea801004962c05657818d45b2058cbfb2f3fb861d7a96720a0a6b3da851cbb
Jul 18 01:58:19.637882 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:19.637864    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:58:20.291915 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:20.291875    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s" event={"ID":"65f40363-9b2c-483a-8d8e-5fad3f554e3b","Type":"ContainerStarted","Data":"52ea801004962c05657818d45b2058cbfb2f3fb861d7a96720a0a6b3da851cbb"}
Jul 18 01:58:21.298497 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:21.298459    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s" event={"ID":"65f40363-9b2c-483a-8d8e-5fad3f554e3b","Type":"ContainerStarted","Data":"583e350b1e3ae956dd19359bef34c4d60582f409467e41c814f5965b270c0266"}
Jul 18 01:58:21.318257 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:21.318191    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s" podStartSLOduration=1.274897147 podStartE2EDuration="2.318171225s" totalImagesPullingTime="1.043274078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:21.314190099 +0000 UTC m=+606.753687030" watchObservedRunningTime="2026-07-18 01:58:21.318171225 +0000 UTC m=+606.757668158"
Jul 18 01:58:22.677395 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:22.677363    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:27.860750 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:27.860713    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"]
Jul 18 01:58:27.864561 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:27.864544    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:27.867004 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:27.866982    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 18 01:58:27.878740 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:27.878718    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"]
Jul 18 01:58:27.963236 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:27.963207    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:27.985717 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:27.985688    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:27.985893 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:27.985758    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/288ccfb5-0784-4828-ae02-9ece0c94864b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:27.985893 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:27.985793    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:27.985893 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:27.985814    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:27.985893 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:27.985831    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt9hb\" (UniqueName: \"kubernetes.io/projected/288ccfb5-0784-4828-ae02-9ece0c94864b-kube-api-access-gt9hb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:27.985893 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:27.985894    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.086839 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.086799    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.086839 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.086840    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.087115 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.086887    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/288ccfb5-0784-4828-ae02-9ece0c94864b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.087115 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.086929    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.087115 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.086953    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.087115 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.086970    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt9hb\" (UniqueName: \"kubernetes.io/projected/288ccfb5-0784-4828-ae02-9ece0c94864b-kube-api-access-gt9hb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.087331 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.087258    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.087331 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.087306    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.087415 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.087330    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.089302 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.089280    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/288ccfb5-0784-4828-ae02-9ece0c94864b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.089476 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.089459    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/288ccfb5-0784-4828-ae02-9ece0c94864b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.094571 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.094549    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt9hb\" (UniqueName: \"kubernetes.io/projected/288ccfb5-0784-4828-ae02-9ece0c94864b-kube-api-access-gt9hb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp\" (UID: \"288ccfb5-0784-4828-ae02-9ece0c94864b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.174865 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.174832    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:28.317364 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.317337    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"]
Jul 18 01:58:28.319598 ip-10-0-128-48 kubenswrapper[2552]: W0718 01:58:28.319569    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod288ccfb5_0784_4828_ae02_9ece0c94864b.slice/crio-ffb2f0342facb9c412ea33cf7fb309f1280b800ac5f2cbc3cb459ce00658f33e WatchSource:0}: Error finding container ffb2f0342facb9c412ea33cf7fb309f1280b800ac5f2cbc3cb459ce00658f33e: Status 404 returned error can't find the container with id ffb2f0342facb9c412ea33cf7fb309f1280b800ac5f2cbc3cb459ce00658f33e
Jul 18 01:58:28.339905 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:28.339875    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp" event={"ID":"288ccfb5-0784-4828-ae02-9ece0c94864b","Type":"ContainerStarted","Data":"ffb2f0342facb9c412ea33cf7fb309f1280b800ac5f2cbc3cb459ce00658f33e"}
Jul 18 01:58:29.347017 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:29.346977    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp" event={"ID":"288ccfb5-0784-4828-ae02-9ece0c94864b","Type":"ContainerStarted","Data":"65245ec08eb6b21610fd7ffa22bc5e8882255e841dff101b451560956f20c393"}
Jul 18 01:58:29.364715 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:29.364665    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp" podStartSLOduration=2.205180465 podStartE2EDuration="2.364649198s" totalImagesPullingTime="159.468733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:29.363802959 +0000 UTC m=+614.803299890" watchObservedRunningTime="2026-07-18 01:58:29.364649198 +0000 UTC m=+614.804146129"
Jul 18 01:58:29.483041 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:29.482991    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:29.483261 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:29.483071    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:29.496019 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:29.495996    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:30.369612 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:30.369582    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c889s"
Jul 18 01:58:34.776453 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:34.776400    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:58:38.175737 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:38.175656    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:38.175737 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:38.175716    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:38.188667 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:38.188638    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:38.410435 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:38.410390    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp"
Jul 18 01:58:51.774884 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:58:51.774852    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:59:24.789346 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:59:24.789311    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 01:59:24.842557 ip-10-0-128-48 kubenswrapper[2552]: I0718 01:59:24.842525    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:04.712308 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:04.712269    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:04.870115 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:04.870081    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:05.831244 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:05.831158    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5b7456c76f-bxb5v"]
Jul 18 02:00:05.831690 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:05.831351    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5b7456c76f-bxb5v" podUID="a747c6ed-93e0-41e4-8e24-b4c660de5b7b" containerName="authorino" containerID="cri-o://f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8" gracePeriod=30
Jul 18 02:00:05.912279 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:05.912245    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:06.006469 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.006409    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:06.105803 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.105776    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b7456c76f-bxb5v"
Jul 18 02:00:06.175379 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.175340    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgvr2\" (UniqueName: \"kubernetes.io/projected/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-kube-api-access-kgvr2\") pod \"a747c6ed-93e0-41e4-8e24-b4c660de5b7b\" (UID: \"a747c6ed-93e0-41e4-8e24-b4c660de5b7b\") "
Jul 18 02:00:06.175379 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.175381    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-tls-cert\") pod \"a747c6ed-93e0-41e4-8e24-b4c660de5b7b\" (UID: \"a747c6ed-93e0-41e4-8e24-b4c660de5b7b\") "
Jul 18 02:00:06.177678 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.177647    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-kube-api-access-kgvr2" (OuterVolumeSpecName: "kube-api-access-kgvr2") pod "a747c6ed-93e0-41e4-8e24-b4c660de5b7b" (UID: "a747c6ed-93e0-41e4-8e24-b4c660de5b7b"). InnerVolumeSpecName "kube-api-access-kgvr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:00:06.186780 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.186752    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a747c6ed-93e0-41e4-8e24-b4c660de5b7b" (UID: "a747c6ed-93e0-41e4-8e24-b4c660de5b7b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:00:06.276275 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.276241    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgvr2\" (UniqueName: \"kubernetes.io/projected/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-kube-api-access-kgvr2\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:00:06.276275 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.276269    2552 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a747c6ed-93e0-41e4-8e24-b4c660de5b7b-tls-cert\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:00:06.904653 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.904620    2552 generic.go:356] "Generic (PLEG): container finished" podID="a747c6ed-93e0-41e4-8e24-b4c660de5b7b" containerID="f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8" exitCode=0
Jul 18 02:00:06.905125 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.904673    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b7456c76f-bxb5v"
Jul 18 02:00:06.905125 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.904706    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b7456c76f-bxb5v" event={"ID":"a747c6ed-93e0-41e4-8e24-b4c660de5b7b","Type":"ContainerDied","Data":"f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8"}
Jul 18 02:00:06.905125 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.904750    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b7456c76f-bxb5v" event={"ID":"a747c6ed-93e0-41e4-8e24-b4c660de5b7b","Type":"ContainerDied","Data":"e64794254c3e4bf76c1dba57efb9612e53ce6335af9c7726d8bec7932155c08c"}
Jul 18 02:00:06.905125 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.904766    2552 scope.go:117] "RemoveContainer" containerID="f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8"
Jul 18 02:00:06.916098 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.916079    2552 scope.go:117] "RemoveContainer" containerID="f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8"
Jul 18 02:00:06.916375 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:00:06.916355    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8\": container with ID starting with f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8 not found: ID does not exist" containerID="f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8"
Jul 18 02:00:06.916472 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.916386    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8"} err="failed to get container status \"f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8\": rpc error: code = NotFound desc = could not find container \"f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8\": container with ID starting with f0e97482bce8c441a176e391dfd433d89d15c9ca8402715e79db420a06d801d8 not found: ID does not exist"
Jul 18 02:00:06.925385 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.925352    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5b7456c76f-bxb5v"]
Jul 18 02:00:06.928336 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:06.928319    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5b7456c76f-bxb5v"]
Jul 18 02:00:07.076100 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:07.076072    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a747c6ed-93e0-41e4-8e24-b4c660de5b7b" path="/var/lib/kubelet/pods/a747c6ed-93e0-41e4-8e24-b4c660de5b7b/volumes"
Jul 18 02:00:44.388990 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:44.388960    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:00:52.272700 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:52.272657    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d774b7794-trxv6"]
Jul 18 02:00:52.273293 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:52.272937    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5d774b7794-trxv6" podUID="3076ffb6-79e5-4d5c-9764-e23e50bad43a" containerName="manager" containerID="cri-o://4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea" gracePeriod=30
Jul 18 02:00:52.533512 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:52.533452    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 02:00:52.569853 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:52.569824    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lrm9\" (UniqueName: \"kubernetes.io/projected/3076ffb6-79e5-4d5c-9764-e23e50bad43a-kube-api-access-5lrm9\") pod \"3076ffb6-79e5-4d5c-9764-e23e50bad43a\" (UID: \"3076ffb6-79e5-4d5c-9764-e23e50bad43a\") "
Jul 18 02:00:52.570041 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:52.569917    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3076ffb6-79e5-4d5c-9764-e23e50bad43a-webhook-cert\") pod \"3076ffb6-79e5-4d5c-9764-e23e50bad43a\" (UID: \"3076ffb6-79e5-4d5c-9764-e23e50bad43a\") "
Jul 18 02:00:52.572126 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:52.572090    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3076ffb6-79e5-4d5c-9764-e23e50bad43a-kube-api-access-5lrm9" (OuterVolumeSpecName: "kube-api-access-5lrm9") pod "3076ffb6-79e5-4d5c-9764-e23e50bad43a" (UID: "3076ffb6-79e5-4d5c-9764-e23e50bad43a"). InnerVolumeSpecName "kube-api-access-5lrm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:00:52.572230 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:52.572140    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3076ffb6-79e5-4d5c-9764-e23e50bad43a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3076ffb6-79e5-4d5c-9764-e23e50bad43a" (UID: "3076ffb6-79e5-4d5c-9764-e23e50bad43a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:00:52.671100 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:52.671063    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lrm9\" (UniqueName: \"kubernetes.io/projected/3076ffb6-79e5-4d5c-9764-e23e50bad43a-kube-api-access-5lrm9\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:00:52.671100 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:52.671095    2552 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3076ffb6-79e5-4d5c-9764-e23e50bad43a-webhook-cert\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:00:53.155730 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:53.155692    2552 generic.go:356] "Generic (PLEG): container finished" podID="3076ffb6-79e5-4d5c-9764-e23e50bad43a" containerID="4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea" exitCode=0
Jul 18 02:00:53.155903 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:53.155738    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d774b7794-trxv6"
Jul 18 02:00:53.155903 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:53.155781    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d774b7794-trxv6" event={"ID":"3076ffb6-79e5-4d5c-9764-e23e50bad43a","Type":"ContainerDied","Data":"4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea"}
Jul 18 02:00:53.155903 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:53.155833    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d774b7794-trxv6" event={"ID":"3076ffb6-79e5-4d5c-9764-e23e50bad43a","Type":"ContainerDied","Data":"29214e7cb11cf30175c70702f03c0b395f9e04938979c2992e2580a0e51a2816"}
Jul 18 02:00:53.155903 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:53.155856    2552 scope.go:117] "RemoveContainer" containerID="4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea"
Jul 18 02:00:53.164940 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:53.164925    2552 scope.go:117] "RemoveContainer" containerID="4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea"
Jul 18 02:00:53.165178 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:00:53.165162    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea\": container with ID starting with 4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea not found: ID does not exist" containerID="4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea"
Jul 18 02:00:53.165242 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:53.165185    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea"} err="failed to get container status \"4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea\": rpc error: code = NotFound desc = could not find container \"4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea\": container with ID starting with 4aed854b0e109c995a81d221595b29cf4a6138405704300432bc02b01a4ac3ea not found: ID does not exist"
Jul 18 02:00:53.170336 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:53.170313    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d774b7794-trxv6"]
Jul 18 02:00:53.174216 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:53.174196    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5d774b7794-trxv6"]
Jul 18 02:00:55.075890 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:00:55.075859    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3076ffb6-79e5-4d5c-9764-e23e50bad43a" path="/var/lib/kubelet/pods/3076ffb6-79e5-4d5c-9764-e23e50bad43a/volumes"
Jul 18 02:01:14.124735 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:14.124705    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:01:54.016898 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.016807    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d774b7794-7s2np"]
Jul 18 02:01:54.017505 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.017256    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a747c6ed-93e0-41e4-8e24-b4c660de5b7b" containerName="authorino"
Jul 18 02:01:54.017505 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.017270    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a747c6ed-93e0-41e4-8e24-b4c660de5b7b" containerName="authorino"
Jul 18 02:01:54.017505 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.017281    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3076ffb6-79e5-4d5c-9764-e23e50bad43a" containerName="manager"
Jul 18 02:01:54.017505 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.017287    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3076ffb6-79e5-4d5c-9764-e23e50bad43a" containerName="manager"
Jul 18 02:01:54.017505 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.017349    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3076ffb6-79e5-4d5c-9764-e23e50bad43a" containerName="manager"
Jul 18 02:01:54.017505 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.017361    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a747c6ed-93e0-41e4-8e24-b4c660de5b7b" containerName="authorino"
Jul 18 02:01:54.020823 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.020805    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:01:54.023445 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.023412    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wb6dd\""
Jul 18 02:01:54.023555 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.023459    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 18 02:01:54.023555 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.023459    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 18 02:01:54.030592 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.030568    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d774b7794-7s2np"]
Jul 18 02:01:54.119353 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.119321    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8372a439-02bc-4077-b7c6-697f26c9a86d-webhook-cert\") pod \"maas-controller-5d774b7794-7s2np\" (UID: \"8372a439-02bc-4077-b7c6-697f26c9a86d\") " pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:01:54.119531 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.119445    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcl8s\" (UniqueName: \"kubernetes.io/projected/8372a439-02bc-4077-b7c6-697f26c9a86d-kube-api-access-wcl8s\") pod \"maas-controller-5d774b7794-7s2np\" (UID: \"8372a439-02bc-4077-b7c6-697f26c9a86d\") " pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:01:54.220733 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.220700    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcl8s\" (UniqueName: \"kubernetes.io/projected/8372a439-02bc-4077-b7c6-697f26c9a86d-kube-api-access-wcl8s\") pod \"maas-controller-5d774b7794-7s2np\" (UID: \"8372a439-02bc-4077-b7c6-697f26c9a86d\") " pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:01:54.220913 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.220755    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8372a439-02bc-4077-b7c6-697f26c9a86d-webhook-cert\") pod \"maas-controller-5d774b7794-7s2np\" (UID: \"8372a439-02bc-4077-b7c6-697f26c9a86d\") " pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:01:54.223224 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.223205    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8372a439-02bc-4077-b7c6-697f26c9a86d-webhook-cert\") pod \"maas-controller-5d774b7794-7s2np\" (UID: \"8372a439-02bc-4077-b7c6-697f26c9a86d\") " pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:01:54.228532 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.228511    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcl8s\" (UniqueName: \"kubernetes.io/projected/8372a439-02bc-4077-b7c6-697f26c9a86d-kube-api-access-wcl8s\") pod \"maas-controller-5d774b7794-7s2np\" (UID: \"8372a439-02bc-4077-b7c6-697f26c9a86d\") " pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:01:54.333064 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.332971    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:01:54.477447 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.477388    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d774b7794-7s2np"]
Jul 18 02:01:54.477584 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:01:54.477546    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8372a439_02bc_4077_b7c6_697f26c9a86d.slice/crio-67a69057122c7fbbfcc6425edcf32502c4b457a8d9e340d8db6f4503ab32018c WatchSource:0}: Error finding container 67a69057122c7fbbfcc6425edcf32502c4b457a8d9e340d8db6f4503ab32018c: Status 404 returned error can't find the container with id 67a69057122c7fbbfcc6425edcf32502c4b457a8d9e340d8db6f4503ab32018c
Jul 18 02:01:54.500250 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:54.500219    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d774b7794-7s2np" event={"ID":"8372a439-02bc-4077-b7c6-697f26c9a86d","Type":"ContainerStarted","Data":"67a69057122c7fbbfcc6425edcf32502c4b457a8d9e340d8db6f4503ab32018c"}
Jul 18 02:01:55.172961 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:55.172932    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:01:55.507715 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:55.507677    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d774b7794-7s2np" event={"ID":"8372a439-02bc-4077-b7c6-697f26c9a86d","Type":"ContainerStarted","Data":"8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff"}
Jul 18 02:01:55.507917 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:55.507901    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:01:55.525181 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:01:55.525121    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5d774b7794-7s2np" podStartSLOduration=1.906081237 podStartE2EDuration="2.525106835s" totalImagesPullingTime="619.025598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:55.522856264 +0000 UTC m=+820.962353209" watchObservedRunningTime="2026-07-18 02:01:55.525106835 +0000 UTC m=+820.964603766"
Jul 18 02:02:06.516936 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:06.516907    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:02:11.080031 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.080004    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:11.176671 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.176634    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d774b7794-7s2np"]
Jul 18 02:02:11.176915 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.176881    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5d774b7794-7s2np" podUID="8372a439-02bc-4077-b7c6-697f26c9a86d" containerName="manager" containerID="cri-o://8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff" gracePeriod=30
Jul 18 02:02:11.437538 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.437511    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:02:11.584552 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.584518    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcl8s\" (UniqueName: \"kubernetes.io/projected/8372a439-02bc-4077-b7c6-697f26c9a86d-kube-api-access-wcl8s\") pod \"8372a439-02bc-4077-b7c6-697f26c9a86d\" (UID: \"8372a439-02bc-4077-b7c6-697f26c9a86d\") "
Jul 18 02:02:11.584732 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.584593    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8372a439-02bc-4077-b7c6-697f26c9a86d-webhook-cert\") pod \"8372a439-02bc-4077-b7c6-697f26c9a86d\" (UID: \"8372a439-02bc-4077-b7c6-697f26c9a86d\") "
Jul 18 02:02:11.586846 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.586814    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8372a439-02bc-4077-b7c6-697f26c9a86d-kube-api-access-wcl8s" (OuterVolumeSpecName: "kube-api-access-wcl8s") pod "8372a439-02bc-4077-b7c6-697f26c9a86d" (UID: "8372a439-02bc-4077-b7c6-697f26c9a86d"). InnerVolumeSpecName "kube-api-access-wcl8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:02:11.586979 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.586954    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8372a439-02bc-4077-b7c6-697f26c9a86d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8372a439-02bc-4077-b7c6-697f26c9a86d" (UID: "8372a439-02bc-4077-b7c6-697f26c9a86d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:02:11.600027 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.599949    2552 generic.go:356] "Generic (PLEG): container finished" podID="8372a439-02bc-4077-b7c6-697f26c9a86d" containerID="8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff" exitCode=0
Jul 18 02:02:11.600027 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.599987    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d774b7794-7s2np"
Jul 18 02:02:11.600191 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.600039    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d774b7794-7s2np" event={"ID":"8372a439-02bc-4077-b7c6-697f26c9a86d","Type":"ContainerDied","Data":"8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff"}
Jul 18 02:02:11.600191 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.600098    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d774b7794-7s2np" event={"ID":"8372a439-02bc-4077-b7c6-697f26c9a86d","Type":"ContainerDied","Data":"67a69057122c7fbbfcc6425edcf32502c4b457a8d9e340d8db6f4503ab32018c"}
Jul 18 02:02:11.600191 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.600129    2552 scope.go:117] "RemoveContainer" containerID="8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff"
Jul 18 02:02:11.609464 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.609417    2552 scope.go:117] "RemoveContainer" containerID="8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff"
Jul 18 02:02:11.609736 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:02:11.609717    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff\": container with ID starting with 8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff not found: ID does not exist" containerID="8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff"
Jul 18 02:02:11.609795 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.609746    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff"} err="failed to get container status \"8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff\": rpc error: code = NotFound desc = could not find container \"8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff\": container with ID starting with 8e5ff5d442d75e342d91a15e23430b90b601ab359684f3405655fd39dc5196ff not found: ID does not exist"
Jul 18 02:02:11.619436 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.619396    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d774b7794-7s2np"]
Jul 18 02:02:11.624050 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.624026    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5d774b7794-7s2np"]
Jul 18 02:02:11.685891 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.685853    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcl8s\" (UniqueName: \"kubernetes.io/projected/8372a439-02bc-4077-b7c6-697f26c9a86d-kube-api-access-wcl8s\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:02:11.685891 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:11.685881    2552 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8372a439-02bc-4077-b7c6-697f26c9a86d-webhook-cert\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:02:12.303206 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.303163    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:12.431999 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.431964    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d774b7794-qksdj"]
Jul 18 02:02:12.432552 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.432523    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8372a439-02bc-4077-b7c6-697f26c9a86d" containerName="manager"
Jul 18 02:02:12.432552 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.432548    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8372a439-02bc-4077-b7c6-697f26c9a86d" containerName="manager"
Jul 18 02:02:12.432732 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.432669    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="8372a439-02bc-4077-b7c6-697f26c9a86d" containerName="manager"
Jul 18 02:02:12.437088 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.437072    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d774b7794-qksdj"
Jul 18 02:02:12.439717 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.439697    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wb6dd\""
Jul 18 02:02:12.439822 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.439700    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 18 02:02:12.439822 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.439705    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 18 02:02:12.443925 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.443902    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d774b7794-qksdj"]
Jul 18 02:02:12.592594 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.592507    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqc8c\" (UniqueName: \"kubernetes.io/projected/e186f0cc-8eb2-41d7-9012-563752dff31a-kube-api-access-wqc8c\") pod \"maas-controller-5d774b7794-qksdj\" (UID: \"e186f0cc-8eb2-41d7-9012-563752dff31a\") " pod="opendatahub/maas-controller-5d774b7794-qksdj"
Jul 18 02:02:12.592594 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.592585    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e186f0cc-8eb2-41d7-9012-563752dff31a-webhook-cert\") pod \"maas-controller-5d774b7794-qksdj\" (UID: \"e186f0cc-8eb2-41d7-9012-563752dff31a\") " pod="opendatahub/maas-controller-5d774b7794-qksdj"
Jul 18 02:02:12.693951 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.693916    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqc8c\" (UniqueName: \"kubernetes.io/projected/e186f0cc-8eb2-41d7-9012-563752dff31a-kube-api-access-wqc8c\") pod \"maas-controller-5d774b7794-qksdj\" (UID: \"e186f0cc-8eb2-41d7-9012-563752dff31a\") " pod="opendatahub/maas-controller-5d774b7794-qksdj"
Jul 18 02:02:12.694133 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.693987    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e186f0cc-8eb2-41d7-9012-563752dff31a-webhook-cert\") pod \"maas-controller-5d774b7794-qksdj\" (UID: \"e186f0cc-8eb2-41d7-9012-563752dff31a\") " pod="opendatahub/maas-controller-5d774b7794-qksdj"
Jul 18 02:02:12.696506 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.696480    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e186f0cc-8eb2-41d7-9012-563752dff31a-webhook-cert\") pod \"maas-controller-5d774b7794-qksdj\" (UID: \"e186f0cc-8eb2-41d7-9012-563752dff31a\") " pod="opendatahub/maas-controller-5d774b7794-qksdj"
Jul 18 02:02:12.701404 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.701379    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqc8c\" (UniqueName: \"kubernetes.io/projected/e186f0cc-8eb2-41d7-9012-563752dff31a-kube-api-access-wqc8c\") pod \"maas-controller-5d774b7794-qksdj\" (UID: \"e186f0cc-8eb2-41d7-9012-563752dff31a\") " pod="opendatahub/maas-controller-5d774b7794-qksdj"
Jul 18 02:02:12.748869 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.748842    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d774b7794-qksdj"
Jul 18 02:02:12.893959 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:12.893930    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d774b7794-qksdj"]
Jul 18 02:02:12.895232 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:02:12.895203    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode186f0cc_8eb2_41d7_9012_563752dff31a.slice/crio-52e6b6f85f6c9fdf155d6a98b7e81de76a3b1341d2f31b2b0ca52b417c72b95c WatchSource:0}: Error finding container 52e6b6f85f6c9fdf155d6a98b7e81de76a3b1341d2f31b2b0ca52b417c72b95c: Status 404 returned error can't find the container with id 52e6b6f85f6c9fdf155d6a98b7e81de76a3b1341d2f31b2b0ca52b417c72b95c
Jul 18 02:02:13.077924 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:13.077884    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8372a439-02bc-4077-b7c6-697f26c9a86d" path="/var/lib/kubelet/pods/8372a439-02bc-4077-b7c6-697f26c9a86d/volumes"
Jul 18 02:02:13.614397 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:13.614299    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d774b7794-qksdj" event={"ID":"e186f0cc-8eb2-41d7-9012-563752dff31a","Type":"ContainerStarted","Data":"6be264080d35796c1ce1522079a33a26f2a978b3b76ce79124f16c383b429a8c"}
Jul 18 02:02:13.614397 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:13.614361    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d774b7794-qksdj" event={"ID":"e186f0cc-8eb2-41d7-9012-563752dff31a","Type":"ContainerStarted","Data":"52e6b6f85f6c9fdf155d6a98b7e81de76a3b1341d2f31b2b0ca52b417c72b95c"}
Jul 18 02:02:13.614848 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:13.614445    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d774b7794-qksdj"
Jul 18 02:02:13.631519 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:13.631465    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5d774b7794-qksdj" podStartSLOduration=1.294633524 podStartE2EDuration="1.631418217s" totalImagesPullingTime="336.784693ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:13.630813192 +0000 UTC m=+839.070310124" watchObservedRunningTime="2026-07-18 02:02:13.631418217 +0000 UTC m=+839.070915150"
Jul 18 02:02:22.357577 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:22.357538    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:24.624101 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:24.624063    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d774b7794-qksdj"
Jul 18 02:02:50.210702 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:50.210657    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:50.309900 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:50.309866    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:50.507561 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:50.507472    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:56.262638 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:56.262605    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:56.324103 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:56.324068    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:56.454316 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:56.454272    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:56.563893 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:56.563809    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:02:56.668002 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:02:56.667963    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:02.253303 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:03:02.253273    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:02.318441 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:03:02.318394    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:03:15.030199 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:03:15.030171    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:03:15.031188 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:03:15.031168    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:04:57.928533 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:04:57.928448    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:06:41.706613 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:06:41.706571    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:07:05.830269 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:05.830223    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"]
Jul 18 02:07:05.830918 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:05.830515    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g" podUID="043e2277-2848-47ca-9564-9b2adec4114e" containerName="manager" containerID="cri-o://afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5" gracePeriod=10
Jul 18 02:07:09.181817 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.181787    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 02:07:09.283012 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.282920    2552 generic.go:356] "Generic (PLEG): container finished" podID="043e2277-2848-47ca-9564-9b2adec4114e" containerID="afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5" exitCode=0
Jul 18 02:07:09.283012 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.282971    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"
Jul 18 02:07:09.283012 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.282993    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g" event={"ID":"043e2277-2848-47ca-9564-9b2adec4114e","Type":"ContainerDied","Data":"afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5"}
Jul 18 02:07:09.283227 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.283066    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g" event={"ID":"043e2277-2848-47ca-9564-9b2adec4114e","Type":"ContainerDied","Data":"deb00cce0fc196428fd5b5be4ae9d8ac47c2203876cfa0102b998d8ae29a95d1"}
Jul 18 02:07:09.283227 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.283095    2552 scope.go:117] "RemoveContainer" containerID="afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5"
Jul 18 02:07:09.293387 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.293368    2552 scope.go:117] "RemoveContainer" containerID="afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5"
Jul 18 02:07:09.293674 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:07:09.293652    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5\": container with ID starting with afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5 not found: ID does not exist" containerID="afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5"
Jul 18 02:07:09.293725 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.293683    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5"} err="failed to get container status \"afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5\": rpc error: code = NotFound desc = could not find container \"afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5\": container with ID starting with afea54f093f5efc4f367f2a0b5707a316c1671b5b294a8ce90a141e2410255e5 not found: ID does not exist"
Jul 18 02:07:09.300048 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.300030    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/043e2277-2848-47ca-9564-9b2adec4114e-extensions-socket-volume\") pod \"043e2277-2848-47ca-9564-9b2adec4114e\" (UID: \"043e2277-2848-47ca-9564-9b2adec4114e\") "
Jul 18 02:07:09.300133 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.300082    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dqx6\" (UniqueName: \"kubernetes.io/projected/043e2277-2848-47ca-9564-9b2adec4114e-kube-api-access-6dqx6\") pod \"043e2277-2848-47ca-9564-9b2adec4114e\" (UID: \"043e2277-2848-47ca-9564-9b2adec4114e\") "
Jul 18 02:07:09.300398 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.300378    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/043e2277-2848-47ca-9564-9b2adec4114e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "043e2277-2848-47ca-9564-9b2adec4114e" (UID: "043e2277-2848-47ca-9564-9b2adec4114e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:07:09.302268 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.302248    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/043e2277-2848-47ca-9564-9b2adec4114e-kube-api-access-6dqx6" (OuterVolumeSpecName: "kube-api-access-6dqx6") pod "043e2277-2848-47ca-9564-9b2adec4114e" (UID: "043e2277-2848-47ca-9564-9b2adec4114e"). InnerVolumeSpecName "kube-api-access-6dqx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:07:09.401110 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.401070    2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/043e2277-2848-47ca-9564-9b2adec4114e-extensions-socket-volume\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:07:09.401110 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.401106    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dqx6\" (UniqueName: \"kubernetes.io/projected/043e2277-2848-47ca-9564-9b2adec4114e-kube-api-access-6dqx6\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:07:09.604623 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.604589    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"]
Jul 18 02:07:09.608024 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:09.607994    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7cz7g"]
Jul 18 02:07:11.076842 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:07:11.076800    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="043e2277-2848-47ca-9564-9b2adec4114e" path="/var/lib/kubelet/pods/043e2277-2848-47ca-9564-9b2adec4114e/volumes"
Jul 18 02:08:07.019397 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.019315    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"]
Jul 18 02:08:07.020037 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.019998    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="043e2277-2848-47ca-9564-9b2adec4114e" containerName="manager"
Jul 18 02:08:07.020037 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.020018    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="043e2277-2848-47ca-9564-9b2adec4114e" containerName="manager"
Jul 18 02:08:07.020149 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.020116    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="043e2277-2848-47ca-9564-9b2adec4114e" containerName="manager"
Jul 18 02:08:07.023289 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.023271    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"
Jul 18 02:08:07.025572 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.025555    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2b4bd\""
Jul 18 02:08:07.035516 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.035490    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"]
Jul 18 02:08:07.098929 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.098886    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/537638b4-9cf1-4748-8d4a-b194fee80bd1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5jndx\" (UID: \"537638b4-9cf1-4748-8d4a-b194fee80bd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"
Jul 18 02:08:07.200326 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.200286    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/537638b4-9cf1-4748-8d4a-b194fee80bd1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5jndx\" (UID: \"537638b4-9cf1-4748-8d4a-b194fee80bd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"
Jul 18 02:08:07.200555 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.200378    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk6sp\" (UniqueName: \"kubernetes.io/projected/537638b4-9cf1-4748-8d4a-b194fee80bd1-kube-api-access-zk6sp\") pod \"kuadrant-operator-controller-manager-86878c769f-5jndx\" (UID: \"537638b4-9cf1-4748-8d4a-b194fee80bd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"
Jul 18 02:08:07.200698 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.200677    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/537638b4-9cf1-4748-8d4a-b194fee80bd1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5jndx\" (UID: \"537638b4-9cf1-4748-8d4a-b194fee80bd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"
Jul 18 02:08:07.300959 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.300871    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6sp\" (UniqueName: \"kubernetes.io/projected/537638b4-9cf1-4748-8d4a-b194fee80bd1-kube-api-access-zk6sp\") pod \"kuadrant-operator-controller-manager-86878c769f-5jndx\" (UID: \"537638b4-9cf1-4748-8d4a-b194fee80bd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"
Jul 18 02:08:07.310080 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.310055    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk6sp\" (UniqueName: \"kubernetes.io/projected/537638b4-9cf1-4748-8d4a-b194fee80bd1-kube-api-access-zk6sp\") pod \"kuadrant-operator-controller-manager-86878c769f-5jndx\" (UID: \"537638b4-9cf1-4748-8d4a-b194fee80bd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"
Jul 18 02:08:07.333801 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.333769    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"
Jul 18 02:08:07.482273 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.482249    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"]
Jul 18 02:08:07.485203 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:08:07.485178    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod537638b4_9cf1_4748_8d4a_b194fee80bd1.slice/crio-fa18ad024c06e8cea289d18c23a936930548ff2d02bc2a57e22ca479ef465a90 WatchSource:0}: Error finding container fa18ad024c06e8cea289d18c23a936930548ff2d02bc2a57e22ca479ef465a90: Status 404 returned error can't find the container with id fa18ad024c06e8cea289d18c23a936930548ff2d02bc2a57e22ca479ef465a90
Jul 18 02:08:07.620379 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.620281    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx" event={"ID":"537638b4-9cf1-4748-8d4a-b194fee80bd1","Type":"ContainerStarted","Data":"85a5ec509c5b2e5c0932bd191147a7c7e73b980c2377d74072cee7469aac7a2b"}
Jul 18 02:08:07.620379 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.620340    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx" event={"ID":"537638b4-9cf1-4748-8d4a-b194fee80bd1","Type":"ContainerStarted","Data":"fa18ad024c06e8cea289d18c23a936930548ff2d02bc2a57e22ca479ef465a90"}
Jul 18 02:08:07.620661 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.620415    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"
Jul 18 02:08:07.638208 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:07.638157    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx" podStartSLOduration=0.638142734 podStartE2EDuration="638.142734ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:08:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:08:07.637721695 +0000 UTC m=+1193.077218651" watchObservedRunningTime="2026-07-18 02:08:07.638142734 +0000 UTC m=+1193.077639665"
Jul 18 02:08:15.073320 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:15.073290    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:08:15.077468 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:15.077442    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:08:18.628307 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:18.628273    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5jndx"
Jul 18 02:08:23.950440 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:23.950390    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:08:24.039087 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:08:24.039042    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:11:49.041374 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:11:49.041338    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:12:38.698623 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:12:38.698579    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:12:38.800223 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:12:38.800191    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:13:15.118813 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:13:15.118782    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:13:15.127279 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:13:15.127255    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:13:40.340218 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:13:40.340125    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:13:51.537508 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:13:51.537473    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:01.113737 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:01.113704    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:09.376069 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:09.376029    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:14.512820 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:14.512783    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:17.489551 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:17.489511    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:17.588663 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:17.588632    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:17.956843 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:17.956812    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:23.747605 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:23.747570    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:23.840830 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:23.840796    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:24.025951 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:24.025869    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:24.351340 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:24.351265    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:25.292467 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:25.292417    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:27.208700 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.208669    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:27.295928 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.295893    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:27.300957 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.300920    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"]
Jul 18 02:14:27.304305 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.304280    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:27.306372 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.306346    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5536ee23-serving-cert\""
Jul 18 02:14:27.314143 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.314119    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"]
Jul 18 02:14:27.368069 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.368032    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:27.447973 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.447938    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a47c61c-19e1-4921-99ed-794d1d4107ba-maas-api-tls\") pod \"maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:27.448160 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.447998    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25rc4\" (UniqueName: \"kubernetes.io/projected/4a47c61c-19e1-4921-99ed-794d1d4107ba-kube-api-access-25rc4\") pod \"maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:27.448160 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.448091    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a47c61c-19e1-4921-99ed-794d1d4107ba-openshift-service-ca\") pod \"maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:27.549295 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.549195    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25rc4\" (UniqueName: \"kubernetes.io/projected/4a47c61c-19e1-4921-99ed-794d1d4107ba-kube-api-access-25rc4\") pod \"maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:27.549295 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.549280    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a47c61c-19e1-4921-99ed-794d1d4107ba-openshift-service-ca\") pod \"maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:27.549593 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.549343    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a47c61c-19e1-4921-99ed-794d1d4107ba-maas-api-tls\") pod \"maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:27.549593 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:14:27.549491    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-serving-cert: secret "maas-api-e2e-ait-5536ee23-serving-cert" not found
Jul 18 02:14:27.549593 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:14:27.549573    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a47c61c-19e1-4921-99ed-794d1d4107ba-maas-api-tls podName:4a47c61c-19e1-4921-99ed-794d1d4107ba nodeName:}" failed. No retries permitted until 2026-07-18 02:14:28.049551306 +0000 UTC m=+1573.489048222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4a47c61c-19e1-4921-99ed-794d1d4107ba-maas-api-tls") pod "maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4" (UID: "4a47c61c-19e1-4921-99ed-794d1d4107ba") : secret "maas-api-e2e-ait-5536ee23-serving-cert" not found
Jul 18 02:14:27.549960 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.549936    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a47c61c-19e1-4921-99ed-794d1d4107ba-openshift-service-ca\") pod \"maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:27.557784 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.557760    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25rc4\" (UniqueName: \"kubernetes.io/projected/4a47c61c-19e1-4921-99ed-794d1d4107ba-kube-api-access-25rc4\") pod \"maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:27.768937 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.768903    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:27.903583 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:27.903549    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:28.004302 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:28.004266    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:14:28.053616 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:28.053576    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a47c61c-19e1-4921-99ed-794d1d4107ba-maas-api-tls\") pod \"maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:28.056214 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:28.056195    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a47c61c-19e1-4921-99ed-794d1d4107ba-maas-api-tls\") pod \"maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:28.218105 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:28.218006    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:28.366962 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:28.366921    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"]
Jul 18 02:14:28.369164 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:14:28.369139    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a47c61c_19e1_4921_99ed_794d1d4107ba.slice/crio-b884c29b2b802524e5afa39a24cbb679bb6b939c5dbfeb07bf9b1d971ba21918 WatchSource:0}: Error finding container b884c29b2b802524e5afa39a24cbb679bb6b939c5dbfeb07bf9b1d971ba21918: Status 404 returned error can't find the container with id b884c29b2b802524e5afa39a24cbb679bb6b939c5dbfeb07bf9b1d971ba21918
Jul 18 02:14:28.370346 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:28.370327    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:14:28.790732 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:28.790697    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4" event={"ID":"4a47c61c-19e1-4921-99ed-794d1d4107ba","Type":"ContainerStarted","Data":"b884c29b2b802524e5afa39a24cbb679bb6b939c5dbfeb07bf9b1d971ba21918"}
Jul 18 02:14:29.797957 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:29.797923    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4" event={"ID":"4a47c61c-19e1-4921-99ed-794d1d4107ba","Type":"ContainerStarted","Data":"ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0"}
Jul 18 02:14:29.798360 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:29.798060    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:29.816874 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:29.816807    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4" podStartSLOduration=2.140267429 podStartE2EDuration="2.816785769s" totalImagesPullingTime="676.51834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:14:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:14:29.814089183 +0000 UTC m=+1575.253586140" watchObservedRunningTime="2026-07-18 02:14:29.816785769 +0000 UTC m=+1575.256282697"
Jul 18 02:14:35.808410 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:35.808378    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:14:52.765220 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:52.765185    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"]
Jul 18 02:14:52.769255 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:52.769236    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"
Jul 18 02:14:52.771964 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:52.771928    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vpdlz\""
Jul 18 02:14:52.773562 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:52.773539    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ebb70c-f3d9-4610-8e7e-1d6576bd0706-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t\" (UID: \"a0ebb70c-f3d9-4610-8e7e-1d6576bd0706\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"
Jul 18 02:14:52.776232 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:52.776204    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"]
Jul 18 02:14:52.874785 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:52.874751    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ebb70c-f3d9-4610-8e7e-1d6576bd0706-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t\" (UID: \"a0ebb70c-f3d9-4610-8e7e-1d6576bd0706\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"
Jul 18 02:14:52.875450 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:52.875403    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ebb70c-f3d9-4610-8e7e-1d6576bd0706-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t\" (UID: \"a0ebb70c-f3d9-4610-8e7e-1d6576bd0706\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"
Jul 18 02:14:53.081988 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:53.081914    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"
Jul 18 02:14:53.229024 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:53.228994    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"]
Jul 18 02:14:53.230914 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:14:53.230873    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0ebb70c_f3d9_4610_8e7e_1d6576bd0706.slice/crio-08e747a32c890d7277b8aa298d0a72c867ac3fd0ee6bcf8f1c6a6e1a48d53d2b WatchSource:0}: Error finding container 08e747a32c890d7277b8aa298d0a72c867ac3fd0ee6bcf8f1c6a6e1a48d53d2b: Status 404 returned error can't find the container with id 08e747a32c890d7277b8aa298d0a72c867ac3fd0ee6bcf8f1c6a6e1a48d53d2b
Jul 18 02:14:53.941365 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:53.941322    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t" event={"ID":"a0ebb70c-f3d9-4610-8e7e-1d6576bd0706","Type":"ContainerStarted","Data":"08e747a32c890d7277b8aa298d0a72c867ac3fd0ee6bcf8f1c6a6e1a48d53d2b"}
Jul 18 02:14:55.954860 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:55.954826    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t" event={"ID":"a0ebb70c-f3d9-4610-8e7e-1d6576bd0706","Type":"ContainerStarted","Data":"4febe29a3dcb238c37e9f713e103a1d6d029c3f13447e057a25d673379d58537"}
Jul 18 02:14:55.972902 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:55.972850    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t" podStartSLOduration=1.855663543 podStartE2EDuration="3.972832409s" totalImagesPullingTime="2.117168866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:14:55.970494926 +0000 UTC m=+1601.409991878" watchObservedRunningTime="2026-07-18 02:14:55.972832409 +0000 UTC m=+1601.412329341"
Jul 18 02:14:56.962598 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:56.962565    2552 generic.go:356] "Generic (PLEG): container finished" podID="a0ebb70c-f3d9-4610-8e7e-1d6576bd0706" containerID="4febe29a3dcb238c37e9f713e103a1d6d029c3f13447e057a25d673379d58537" exitCode=0
Jul 18 02:14:56.963005 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:56.962631    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t" event={"ID":"a0ebb70c-f3d9-4610-8e7e-1d6576bd0706","Type":"ContainerDied","Data":"4febe29a3dcb238c37e9f713e103a1d6d029c3f13447e057a25d673379d58537"}
Jul 18 02:14:58.116497 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:58.116467    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"
Jul 18 02:14:58.226200 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:58.226113    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ebb70c-f3d9-4610-8e7e-1d6576bd0706-maas-api-service-ca\") pod \"a0ebb70c-f3d9-4610-8e7e-1d6576bd0706\" (UID: \"a0ebb70c-f3d9-4610-8e7e-1d6576bd0706\") "
Jul 18 02:14:58.226549 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:58.226524    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0ebb70c-f3d9-4610-8e7e-1d6576bd0706-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a0ebb70c-f3d9-4610-8e7e-1d6576bd0706" (UID: "a0ebb70c-f3d9-4610-8e7e-1d6576bd0706"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:14:58.326741 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:58.326698    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ebb70c-f3d9-4610-8e7e-1d6576bd0706-maas-api-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:14:58.981605 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:58.981578    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"
Jul 18 02:14:58.981605 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:58.981594    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t" event={"ID":"a0ebb70c-f3d9-4610-8e7e-1d6576bd0706","Type":"ContainerDied","Data":"08e747a32c890d7277b8aa298d0a72c867ac3fd0ee6bcf8f1c6a6e1a48d53d2b"}
Jul 18 02:14:58.981821 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:14:58.981637    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08e747a32c890d7277b8aa298d0a72c867ac3fd0ee6bcf8f1c6a6e1a48d53d2b"
Jul 18 02:15:00.149967 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.149934    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5"]
Jul 18 02:15:00.150372 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.150359    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0ebb70c-f3d9-4610-8e7e-1d6576bd0706" containerName="revoke-keys"
Jul 18 02:15:00.150450 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.150374    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0ebb70c-f3d9-4610-8e7e-1d6576bd0706" containerName="revoke-keys"
Jul 18 02:15:00.150500 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.150475    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0ebb70c-f3d9-4610-8e7e-1d6576bd0706" containerName="revoke-keys"
Jul 18 02:15:00.153575 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.153555    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z"]
Jul 18 02:15:00.153693 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.153678    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5"
Jul 18 02:15:00.156372 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.156354    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vpdlz\""
Jul 18 02:15:00.156950 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.156937    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z"
Jul 18 02:15:00.162333 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.162309    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5"]
Jul 18 02:15:00.166459 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.166438    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z"]
Jul 18 02:15:00.169717 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.169697    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5"
Jul 18 02:15:00.177262 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.177239    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z"
Jul 18 02:15:00.325240 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.325172    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5"]
Jul 18 02:15:00.327778 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:15:00.327745    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c53420d_3de7_41b2_a1f9_9eaa13637dcf.slice/crio-00427cdbf110a3634de3951f0948475373882b5ce4e63733ea613384abb69922 WatchSource:0}: Error finding container 00427cdbf110a3634de3951f0948475373882b5ce4e63733ea613384abb69922: Status 404 returned error can't find the container with id 00427cdbf110a3634de3951f0948475373882b5ce4e63733ea613384abb69922
Jul 18 02:15:00.557817 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.557788    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z"]
Jul 18 02:15:00.559568 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:15:00.559539    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e11beb_5f63_4513_9126_b32af21dc2b0.slice/crio-044e70443b7858f0c6b4d4203217ee17e59e12b40dbb04d501732a835736e375 WatchSource:0}: Error finding container 044e70443b7858f0c6b4d4203217ee17e59e12b40dbb04d501732a835736e375: Status 404 returned error can't find the container with id 044e70443b7858f0c6b4d4203217ee17e59e12b40dbb04d501732a835736e375
Jul 18 02:15:00.993970 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.993870    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5" event={"ID":"8c53420d-3de7-41b2-a1f9-9eaa13637dcf","Type":"ContainerStarted","Data":"005764b4f254656912de7addf7ebda52e9c4bfa54162c8497175f534077dc979"}
Jul 18 02:15:00.993970 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.993933    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5" event={"ID":"8c53420d-3de7-41b2-a1f9-9eaa13637dcf","Type":"ContainerStarted","Data":"00427cdbf110a3634de3951f0948475373882b5ce4e63733ea613384abb69922"}
Jul 18 02:15:00.995301 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.995272    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z" event={"ID":"36e11beb-5f63-4513-9126-b32af21dc2b0","Type":"ContainerStarted","Data":"89b5f033f839c43f54ec82741ee7e88a626fc941921c0efe9d4f30c709e2af6c"}
Jul 18 02:15:00.995455 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:00.995318    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z" event={"ID":"36e11beb-5f63-4513-9126-b32af21dc2b0","Type":"ContainerStarted","Data":"044e70443b7858f0c6b4d4203217ee17e59e12b40dbb04d501732a835736e375"}
Jul 18 02:15:01.009610 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:01.009559    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5" podStartSLOduration=1.009541934 podStartE2EDuration="1.009541934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:15:01.007892923 +0000 UTC m=+1606.447389855" watchObservedRunningTime="2026-07-18 02:15:01.009541934 +0000 UTC m=+1606.449038866"
Jul 18 02:15:01.023227 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:01.023179    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z" podStartSLOduration=1.023162027 podStartE2EDuration="1.023162027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:15:01.021645553 +0000 UTC m=+1606.461142484" watchObservedRunningTime="2026-07-18 02:15:01.023162027 +0000 UTC m=+1606.462658986"
Jul 18 02:15:02.005133 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:02.005097    2552 generic.go:356] "Generic (PLEG): container finished" podID="8c53420d-3de7-41b2-a1f9-9eaa13637dcf" containerID="005764b4f254656912de7addf7ebda52e9c4bfa54162c8497175f534077dc979" exitCode=0
Jul 18 02:15:02.005589 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:02.005192    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5" event={"ID":"8c53420d-3de7-41b2-a1f9-9eaa13637dcf","Type":"ContainerDied","Data":"005764b4f254656912de7addf7ebda52e9c4bfa54162c8497175f534077dc979"}
Jul 18 02:15:02.006640 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:02.006614    2552 generic.go:356] "Generic (PLEG): container finished" podID="36e11beb-5f63-4513-9126-b32af21dc2b0" containerID="89b5f033f839c43f54ec82741ee7e88a626fc941921c0efe9d4f30c709e2af6c" exitCode=0
Jul 18 02:15:02.006764 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:02.006648    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z" event={"ID":"36e11beb-5f63-4513-9126-b32af21dc2b0","Type":"ContainerDied","Data":"89b5f033f839c43f54ec82741ee7e88a626fc941921c0efe9d4f30c709e2af6c"}
Jul 18 02:15:03.178011 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.177977    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5"
Jul 18 02:15:03.182831 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.182807    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z"
Jul 18 02:15:03.204489 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.204458    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"]
Jul 18 02:15:03.204975 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.204934    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4" podUID="4a47c61c-19e1-4921-99ed-794d1d4107ba" containerName="maas-api" containerID="cri-o://ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0" gracePeriod=30
Jul 18 02:15:03.218024 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.217994    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z"]
Jul 18 02:15:03.219520 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.219482    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:03.477632 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.477603    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:15:03.576240 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.576136    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a47c61c-19e1-4921-99ed-794d1d4107ba-maas-api-tls\") pod \"4a47c61c-19e1-4921-99ed-794d1d4107ba\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") "
Jul 18 02:15:03.576240 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.576208    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25rc4\" (UniqueName: \"kubernetes.io/projected/4a47c61c-19e1-4921-99ed-794d1d4107ba-kube-api-access-25rc4\") pod \"4a47c61c-19e1-4921-99ed-794d1d4107ba\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") "
Jul 18 02:15:03.576543 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.576256    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a47c61c-19e1-4921-99ed-794d1d4107ba-openshift-service-ca\") pod \"4a47c61c-19e1-4921-99ed-794d1d4107ba\" (UID: \"4a47c61c-19e1-4921-99ed-794d1d4107ba\") "
Jul 18 02:15:03.576777 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.576742    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a47c61c-19e1-4921-99ed-794d1d4107ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4a47c61c-19e1-4921-99ed-794d1d4107ba" (UID: "4a47c61c-19e1-4921-99ed-794d1d4107ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:15:03.578678 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.578657    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a47c61c-19e1-4921-99ed-794d1d4107ba-kube-api-access-25rc4" (OuterVolumeSpecName: "kube-api-access-25rc4") pod "4a47c61c-19e1-4921-99ed-794d1d4107ba" (UID: "4a47c61c-19e1-4921-99ed-794d1d4107ba"). InnerVolumeSpecName "kube-api-access-25rc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:15:03.578755 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.578688    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a47c61c-19e1-4921-99ed-794d1d4107ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4a47c61c-19e1-4921-99ed-794d1d4107ba" (UID: "4a47c61c-19e1-4921-99ed-794d1d4107ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:15:03.677829 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.677791    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a47c61c-19e1-4921-99ed-794d1d4107ba-openshift-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:15:03.677829 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.677821    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a47c61c-19e1-4921-99ed-794d1d4107ba-maas-api-tls\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:15:03.677829 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:03.677833    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25rc4\" (UniqueName: \"kubernetes.io/projected/4a47c61c-19e1-4921-99ed-794d1d4107ba-kube-api-access-25rc4\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:15:04.021687 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.021650    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5"
Jul 18 02:15:04.021871 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.021654    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-9flw5" event={"ID":"8c53420d-3de7-41b2-a1f9-9eaa13637dcf","Type":"ContainerDied","Data":"00427cdbf110a3634de3951f0948475373882b5ce4e63733ea613384abb69922"}
Jul 18 02:15:04.021871 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.021803    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00427cdbf110a3634de3951f0948475373882b5ce4e63733ea613384abb69922"
Jul 18 02:15:04.023099 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.023078    2552 generic.go:356] "Generic (PLEG): container finished" podID="4a47c61c-19e1-4921-99ed-794d1d4107ba" containerID="ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0" exitCode=0
Jul 18 02:15:04.023207 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.023121    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"
Jul 18 02:15:04.023207 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.023149    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4" event={"ID":"4a47c61c-19e1-4921-99ed-794d1d4107ba","Type":"ContainerDied","Data":"ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0"}
Jul 18 02:15:04.023313 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.023211    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4" event={"ID":"4a47c61c-19e1-4921-99ed-794d1d4107ba","Type":"ContainerDied","Data":"b884c29b2b802524e5afa39a24cbb679bb6b939c5dbfeb07bf9b1d971ba21918"}
Jul 18 02:15:04.023313 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.023242    2552 scope.go:117] "RemoveContainer" containerID="ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0"
Jul 18 02:15:04.024645 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.024616    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z"
Jul 18 02:15:04.024707 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.024644    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z" event={"ID":"36e11beb-5f63-4513-9126-b32af21dc2b0","Type":"ContainerDied","Data":"044e70443b7858f0c6b4d4203217ee17e59e12b40dbb04d501732a835736e375"}
Jul 18 02:15:04.024707 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.024691    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="044e70443b7858f0c6b4d4203217ee17e59e12b40dbb04d501732a835736e375"
Jul 18 02:15:04.034835 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.034821    2552 scope.go:117] "RemoveContainer" containerID="ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0"
Jul 18 02:15:04.035112 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:15:04.035094    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0\": container with ID starting with ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0 not found: ID does not exist" containerID="ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0"
Jul 18 02:15:04.035166 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.035120    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0"} err="failed to get container status \"ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0\": rpc error: code = NotFound desc = could not find container \"ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0\": container with ID starting with ee55f6c61165c75eaa070c8b0d15f9af57c5180dc995b3b90719644f5a11b1a0 not found: ID does not exist"
Jul 18 02:15:04.053723 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.053699    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"]
Jul 18 02:15:04.057783 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.057761    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5536ee23-d5d7884d8-mq7d4"]
Jul 18 02:15:04.065144 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.065119    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z"]
Jul 18 02:15:04.068947 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:04.068918    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5536ee23-29739015-ld98z"]
Jul 18 02:15:05.077893 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:05.077857    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36e11beb-5f63-4513-9126-b32af21dc2b0" path="/var/lib/kubelet/pods/36e11beb-5f63-4513-9126-b32af21dc2b0/volumes"
Jul 18 02:15:05.078384 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:05.078349    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a47c61c-19e1-4921-99ed-794d1d4107ba" path="/var/lib/kubelet/pods/4a47c61c-19e1-4921-99ed-794d1d4107ba/volumes"
Jul 18 02:15:07.845486 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:07.845455    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"]
Jul 18 02:15:07.845967 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:07.845715    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:07.848078 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:07.848058    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5536ee23-rkb9t"]
Jul 18 02:15:08.020907 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:08.020879    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:08.274400 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:08.274358    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:08.575830 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:08.575708    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:08.707055 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:08.707020    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:08.796396 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:08.796361    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:09.076013 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:09.075980    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0ebb70c-f3d9-4610-8e7e-1d6576bd0706" path="/var/lib/kubelet/pods/a0ebb70c-f3d9-4610-8e7e-1d6576bd0706/volumes"
Jul 18 02:15:09.141335 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:09.141279    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:09.486432 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:09.486386    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:11.447066 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.447019    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:11.590314 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.590264    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"]
Jul 18 02:15:11.590951 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.590924    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e11beb-5f63-4513-9126-b32af21dc2b0" containerName="cleanup"
Jul 18 02:15:11.591135 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.591112    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e11beb-5f63-4513-9126-b32af21dc2b0" containerName="cleanup"
Jul 18 02:15:11.591263 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.591151    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a47c61c-19e1-4921-99ed-794d1d4107ba" containerName="maas-api"
Jul 18 02:15:11.591263 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.591169    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a47c61c-19e1-4921-99ed-794d1d4107ba" containerName="maas-api"
Jul 18 02:15:11.591263 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.591185    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c53420d-3de7-41b2-a1f9-9eaa13637dcf" containerName="cleanup"
Jul 18 02:15:11.591263 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.591193    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c53420d-3de7-41b2-a1f9-9eaa13637dcf" containerName="cleanup"
Jul 18 02:15:11.591409 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.591314    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e11beb-5f63-4513-9126-b32af21dc2b0" containerName="cleanup"
Jul 18 02:15:11.591409 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.591331    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c53420d-3de7-41b2-a1f9-9eaa13637dcf" containerName="cleanup"
Jul 18 02:15:11.591409 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.591345    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a47c61c-19e1-4921-99ed-794d1d4107ba" containerName="maas-api"
Jul 18 02:15:11.596883 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.596839    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:11.603474 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.603450    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-8f8edd4c-serving-cert\""
Jul 18 02:15:11.610133 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.610105    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"]
Jul 18 02:15:11.625919 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.625893    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:11.651197 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.651161    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/799d3522-cc88-4564-bc0c-701198e8875b-openshift-service-ca\") pod \"maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:11.651197 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.651201    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcz2f\" (UniqueName: \"kubernetes.io/projected/799d3522-cc88-4564-bc0c-701198e8875b-kube-api-access-qcz2f\") pod \"maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:11.651477 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.651293    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/799d3522-cc88-4564-bc0c-701198e8875b-maas-api-tls\") pod \"maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:11.752541 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.752439    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/799d3522-cc88-4564-bc0c-701198e8875b-maas-api-tls\") pod \"maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:11.752541 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.752522    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/799d3522-cc88-4564-bc0c-701198e8875b-openshift-service-ca\") pod \"maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:11.752742 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.752548    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcz2f\" (UniqueName: \"kubernetes.io/projected/799d3522-cc88-4564-bc0c-701198e8875b-kube-api-access-qcz2f\") pod \"maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:11.753316 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.753217    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/799d3522-cc88-4564-bc0c-701198e8875b-openshift-service-ca\") pod \"maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:11.755051 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.755028    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/799d3522-cc88-4564-bc0c-701198e8875b-maas-api-tls\") pod \"maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:11.760632 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.760609    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcz2f\" (UniqueName: \"kubernetes.io/projected/799d3522-cc88-4564-bc0c-701198e8875b-kube-api-access-qcz2f\") pod \"maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:11.771858 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.771833    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:11.915701 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:11.915667    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:12.064679 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:12.064649    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"]
Jul 18 02:15:12.066322 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:15:12.066299    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod799d3522_cc88_4564_bc0c_701198e8875b.slice/crio-a874fe3e138f3b040bcd6902d1b4626254c7221c5053346d50dd7f73b69610d7 WatchSource:0}: Error finding container a874fe3e138f3b040bcd6902d1b4626254c7221c5053346d50dd7f73b69610d7: Status 404 returned error can't find the container with id a874fe3e138f3b040bcd6902d1b4626254c7221c5053346d50dd7f73b69610d7
Jul 18 02:15:12.074831 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:12.074805    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx" event={"ID":"799d3522-cc88-4564-bc0c-701198e8875b","Type":"ContainerStarted","Data":"a874fe3e138f3b040bcd6902d1b4626254c7221c5053346d50dd7f73b69610d7"}
Jul 18 02:15:12.083969 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:12.083938    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:12.184349 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:12.184308    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:13.082863 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:13.082828    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx" event={"ID":"799d3522-cc88-4564-bc0c-701198e8875b","Type":"ContainerStarted","Data":"7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b"}
Jul 18 02:15:13.083283 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:13.082957    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:13.100150 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:13.100101    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx" podStartSLOduration=1.696725099 podStartE2EDuration="2.100086283s" totalImagesPullingTime="403.361184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:15:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:15:13.097634644 +0000 UTC m=+1618.537131605" watchObservedRunningTime="2026-07-18 02:15:13.100086283 +0000 UTC m=+1618.539583216"
Jul 18 02:15:19.094514 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:19.094485    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:36.506562 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:36.506520    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"]
Jul 18 02:15:36.510482 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:36.510456    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"
Jul 18 02:15:36.512683 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:36.512661    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vpdlz\""
Jul 18 02:15:36.519592 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:36.519488    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"]
Jul 18 02:15:36.586617 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:36.586545    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks\" (UID: \"5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"
Jul 18 02:15:36.687098 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:36.687056    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks\" (UID: \"5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"
Jul 18 02:15:36.687779 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:36.687747    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks\" (UID: \"5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"
Jul 18 02:15:36.824597 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:36.824515    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"
Jul 18 02:15:36.969771 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:15:36.969740    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2b6482_d4e3_4cfd_bc97_edd6ae08c2e5.slice/crio-04cb4b9435504e339bbb3f76996758b6e3a1c703694ecbc3773e57adaf3d54a1 WatchSource:0}: Error finding container 04cb4b9435504e339bbb3f76996758b6e3a1c703694ecbc3773e57adaf3d54a1: Status 404 returned error can't find the container with id 04cb4b9435504e339bbb3f76996758b6e3a1c703694ecbc3773e57adaf3d54a1
Jul 18 02:15:36.969771 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:36.969743    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"]
Jul 18 02:15:37.228631 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:37.228605    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks" event={"ID":"5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5","Type":"ContainerStarted","Data":"66c1dc37275a6f6da3d3dc3075ccdadb9c6bf32b9209860f8eab9707b1175178"}
Jul 18 02:15:37.228778 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:37.228650    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks" event={"ID":"5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5","Type":"ContainerStarted","Data":"04cb4b9435504e339bbb3f76996758b6e3a1c703694ecbc3773e57adaf3d54a1"}
Jul 18 02:15:37.244414 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:37.244361    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks" podStartSLOduration=1.244345856 podStartE2EDuration="1.244345856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:15:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:15:37.242544352 +0000 UTC m=+1642.682041309" watchObservedRunningTime="2026-07-18 02:15:37.244345856 +0000 UTC m=+1642.683842788"
Jul 18 02:15:38.236486 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:38.236417    2552 generic.go:356] "Generic (PLEG): container finished" podID="5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5" containerID="66c1dc37275a6f6da3d3dc3075ccdadb9c6bf32b9209860f8eab9707b1175178" exitCode=0
Jul 18 02:15:38.236873 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:38.236502    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks" event={"ID":"5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5","Type":"ContainerDied","Data":"66c1dc37275a6f6da3d3dc3075ccdadb9c6bf32b9209860f8eab9707b1175178"}
Jul 18 02:15:39.378680 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:39.378655    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"
Jul 18 02:15:39.405796 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:39.405766    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5-maas-api-service-ca\") pod \"5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5\" (UID: \"5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5\") "
Jul 18 02:15:39.406068 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:39.406045    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5" (UID: "5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:15:39.506633 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:39.506553    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5-maas-api-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:15:40.249508 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:40.249460    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks" event={"ID":"5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5","Type":"ContainerDied","Data":"04cb4b9435504e339bbb3f76996758b6e3a1c703694ecbc3773e57adaf3d54a1"}
Jul 18 02:15:40.249695 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:40.249525    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04cb4b9435504e339bbb3f76996758b6e3a1c703694ecbc3773e57adaf3d54a1"
Jul 18 02:15:40.249695 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:40.249477    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"
Jul 18 02:15:46.893036 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:46.892991    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"]
Jul 18 02:15:46.893535 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:46.893246    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx" podUID="799d3522-cc88-4564-bc0c-701198e8875b" containerName="maas-api" containerID="cri-o://7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b" gracePeriod=30
Jul 18 02:15:46.920808 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:46.920781    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:47.154792 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.154720    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:47.176716 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.176688    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/799d3522-cc88-4564-bc0c-701198e8875b-maas-api-tls\") pod \"799d3522-cc88-4564-bc0c-701198e8875b\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") "
Jul 18 02:15:47.176891 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.176814    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/799d3522-cc88-4564-bc0c-701198e8875b-openshift-service-ca\") pod \"799d3522-cc88-4564-bc0c-701198e8875b\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") "
Jul 18 02:15:47.176891 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.176844    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcz2f\" (UniqueName: \"kubernetes.io/projected/799d3522-cc88-4564-bc0c-701198e8875b-kube-api-access-qcz2f\") pod \"799d3522-cc88-4564-bc0c-701198e8875b\" (UID: \"799d3522-cc88-4564-bc0c-701198e8875b\") "
Jul 18 02:15:47.177231 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.177198    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/799d3522-cc88-4564-bc0c-701198e8875b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "799d3522-cc88-4564-bc0c-701198e8875b" (UID: "799d3522-cc88-4564-bc0c-701198e8875b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:15:47.179346 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.179315    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/799d3522-cc88-4564-bc0c-701198e8875b-kube-api-access-qcz2f" (OuterVolumeSpecName: "kube-api-access-qcz2f") pod "799d3522-cc88-4564-bc0c-701198e8875b" (UID: "799d3522-cc88-4564-bc0c-701198e8875b"). InnerVolumeSpecName "kube-api-access-qcz2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:15:47.179346 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.179327    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/799d3522-cc88-4564-bc0c-701198e8875b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "799d3522-cc88-4564-bc0c-701198e8875b" (UID: "799d3522-cc88-4564-bc0c-701198e8875b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:15:47.277682 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.277646    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/799d3522-cc88-4564-bc0c-701198e8875b-openshift-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:15:47.277682 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.277676    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcz2f\" (UniqueName: \"kubernetes.io/projected/799d3522-cc88-4564-bc0c-701198e8875b-kube-api-access-qcz2f\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:15:47.277682 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.277686    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/799d3522-cc88-4564-bc0c-701198e8875b-maas-api-tls\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:15:47.291912 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.291884    2552 generic.go:356] "Generic (PLEG): container finished" podID="799d3522-cc88-4564-bc0c-701198e8875b" containerID="7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b" exitCode=0
Jul 18 02:15:47.292082 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.291935    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"
Jul 18 02:15:47.292082 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.291979    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx" event={"ID":"799d3522-cc88-4564-bc0c-701198e8875b","Type":"ContainerDied","Data":"7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b"}
Jul 18 02:15:47.292082 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.292034    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx" event={"ID":"799d3522-cc88-4564-bc0c-701198e8875b","Type":"ContainerDied","Data":"a874fe3e138f3b040bcd6902d1b4626254c7221c5053346d50dd7f73b69610d7"}
Jul 18 02:15:47.292082 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.292063    2552 scope.go:117] "RemoveContainer" containerID="7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b"
Jul 18 02:15:47.302272 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.302255    2552 scope.go:117] "RemoveContainer" containerID="7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b"
Jul 18 02:15:47.302571 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:15:47.302545    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b\": container with ID starting with 7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b not found: ID does not exist" containerID="7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b"
Jul 18 02:15:47.302643 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.302585    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b"} err="failed to get container status \"7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b\": rpc error: code = NotFound desc = could not find container \"7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b\": container with ID starting with 7029fb638f085a8eed38ce356e78478dd7905e7ff7a96d54b4178fbb5137e11b not found: ID does not exist"
Jul 18 02:15:47.312944 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.312920    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"]
Jul 18 02:15:47.316320 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:47.316300    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f8edd4c-799f9c55f7-c4xrx"]
Jul 18 02:15:49.076797 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:49.076761    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="799d3522-cc88-4564-bc0c-701198e8875b" path="/var/lib/kubelet/pods/799d3522-cc88-4564-bc0c-701198e8875b/volumes"
Jul 18 02:15:51.571414 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:51.571374    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:51.580031 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:51.580005    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"]
Jul 18 02:15:51.582322 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:51.582300    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f8edd4c-mskks"]
Jul 18 02:15:51.756277 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:51.756230    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:52.000820 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:52.000781    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:52.236063 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:52.236018    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:52.570966 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:52.570931    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:52.925349 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:52.925315    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:53.077977 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:53.077941    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5" path="/var/lib/kubelet/pods/5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5/volumes"
Jul 18 02:15:54.630058 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.630019    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:54.728508 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.728466    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:54.866660 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.866618    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"]
Jul 18 02:15:54.867372 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.867353    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="799d3522-cc88-4564-bc0c-701198e8875b" containerName="maas-api"
Jul 18 02:15:54.867444 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.867378    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="799d3522-cc88-4564-bc0c-701198e8875b" containerName="maas-api"
Jul 18 02:15:54.867495 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.867443    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5" containerName="revoke-keys"
Jul 18 02:15:54.867495 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.867454    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5" containerName="revoke-keys"
Jul 18 02:15:54.867590 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.867577    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e2b6482-d4e3-4cfd-bc97-edd6ae08c2e5" containerName="revoke-keys"
Jul 18 02:15:54.867622 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.867597    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="799d3522-cc88-4564-bc0c-701198e8875b" containerName="maas-api"
Jul 18 02:15:54.872586 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.872564    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:54.877451 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.875209    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-92aad9cd-serving-cert\""
Jul 18 02:15:54.888223 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.888153    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"]
Jul 18 02:15:54.902054 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.902019    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:54.949158 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.949118    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b07dd288-c6b9-4630-811c-cf89c91d99a6-maas-api-tls\") pod \"maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:54.949347 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.949168    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b07dd288-c6b9-4630-811c-cf89c91d99a6-openshift-service-ca\") pod \"maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:54.949347 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:54.949192    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbs7q\" (UniqueName: \"kubernetes.io/projected/b07dd288-c6b9-4630-811c-cf89c91d99a6-kube-api-access-rbs7q\") pod \"maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:55.049716 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.049676    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b07dd288-c6b9-4630-811c-cf89c91d99a6-maas-api-tls\") pod \"maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:55.049932 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.049734    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b07dd288-c6b9-4630-811c-cf89c91d99a6-openshift-service-ca\") pod \"maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:55.049932 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.049767    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbs7q\" (UniqueName: \"kubernetes.io/projected/b07dd288-c6b9-4630-811c-cf89c91d99a6-kube-api-access-rbs7q\") pod \"maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:55.049932 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:15:55.049842    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-serving-cert: secret "maas-api-e2e-ait-92aad9cd-serving-cert" not found
Jul 18 02:15:55.049932 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:15:55.049920    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07dd288-c6b9-4630-811c-cf89c91d99a6-maas-api-tls podName:b07dd288-c6b9-4630-811c-cf89c91d99a6 nodeName:}" failed. No retries permitted until 2026-07-18 02:15:55.549903558 +0000 UTC m=+1660.989400468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b07dd288-c6b9-4630-811c-cf89c91d99a6-maas-api-tls") pod "maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp" (UID: "b07dd288-c6b9-4630-811c-cf89c91d99a6") : secret "maas-api-e2e-ait-92aad9cd-serving-cert" not found
Jul 18 02:15:55.050377 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.050357    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b07dd288-c6b9-4630-811c-cf89c91d99a6-openshift-service-ca\") pod \"maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:55.059714 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.059689    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbs7q\" (UniqueName: \"kubernetes.io/projected/b07dd288-c6b9-4630-811c-cf89c91d99a6-kube-api-access-rbs7q\") pod \"maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:55.243592 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.243561    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:55.500349 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.500259    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:55.556550 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.556513    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b07dd288-c6b9-4630-811c-cf89c91d99a6-maas-api-tls\") pod \"maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:55.559324 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.559296    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b07dd288-c6b9-4630-811c-cf89c91d99a6-maas-api-tls\") pod \"maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:55.598661 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.598630    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:15:55.794662 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.794576    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:55.945735 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:55.945705    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"]
Jul 18 02:15:55.947723 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:15:55.947697    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07dd288_c6b9_4630_811c_cf89c91d99a6.slice/crio-644690b443c4b459f41194c3399fc38e351efd8f538a40f48c97e1e0136035b3 WatchSource:0}: Error finding container 644690b443c4b459f41194c3399fc38e351efd8f538a40f48c97e1e0136035b3: Status 404 returned error can't find the container with id 644690b443c4b459f41194c3399fc38e351efd8f538a40f48c97e1e0136035b3
Jul 18 02:15:56.351669 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:56.351625    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp" event={"ID":"b07dd288-c6b9-4630-811c-cf89c91d99a6","Type":"ContainerStarted","Data":"644690b443c4b459f41194c3399fc38e351efd8f538a40f48c97e1e0136035b3"}
Jul 18 02:15:57.359031 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:57.358998    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp" event={"ID":"b07dd288-c6b9-4630-811c-cf89c91d99a6","Type":"ContainerStarted","Data":"826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6"}
Jul 18 02:15:57.359410 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:57.359069    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:15:57.377833 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:15:57.377779    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp" podStartSLOduration=2.961716273 podStartE2EDuration="3.377762307s" totalImagesPullingTime="416.046034ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:15:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:15:57.376464852 +0000 UTC m=+1662.815961786" watchObservedRunningTime="2026-07-18 02:15:57.377762307 +0000 UTC m=+1662.817259239"
Jul 18 02:16:03.375013 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:03.374987    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:16:20.456943 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:20.456909    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"]
Jul 18 02:16:20.462012 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:20.461986    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"
Jul 18 02:16:20.464331 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:20.464299    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vpdlz\""
Jul 18 02:16:20.466679 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:20.466659    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"]
Jul 18 02:16:20.593030 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:20.592988    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e03c858a-a1c2-4e7f-8dbf-74a63a7dab22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd\" (UID: \"e03c858a-a1c2-4e7f-8dbf-74a63a7dab22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"
Jul 18 02:16:20.693967 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:20.693927    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e03c858a-a1c2-4e7f-8dbf-74a63a7dab22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd\" (UID: \"e03c858a-a1c2-4e7f-8dbf-74a63a7dab22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"
Jul 18 02:16:20.694595 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:20.694576    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e03c858a-a1c2-4e7f-8dbf-74a63a7dab22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd\" (UID: \"e03c858a-a1c2-4e7f-8dbf-74a63a7dab22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"
Jul 18 02:16:20.776668 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:20.776582    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"
Jul 18 02:16:20.927179 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:20.927145    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"]
Jul 18 02:16:20.928789 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:16:20.928760    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode03c858a_a1c2_4e7f_8dbf_74a63a7dab22.slice/crio-578359b4731bad63339e3ca6b06d1b07871f4068ceef72fb04fb41548c978191 WatchSource:0}: Error finding container 578359b4731bad63339e3ca6b06d1b07871f4068ceef72fb04fb41548c978191: Status 404 returned error can't find the container with id 578359b4731bad63339e3ca6b06d1b07871f4068ceef72fb04fb41548c978191
Jul 18 02:16:21.510729 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:21.510696    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd" event={"ID":"e03c858a-a1c2-4e7f-8dbf-74a63a7dab22","Type":"ContainerStarted","Data":"26071a0605328aee3eb29625bb2863e0631b9aba6147e4b01a0362b7939062a4"}
Jul 18 02:16:21.511161 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:21.510756    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd" event={"ID":"e03c858a-a1c2-4e7f-8dbf-74a63a7dab22","Type":"ContainerStarted","Data":"578359b4731bad63339e3ca6b06d1b07871f4068ceef72fb04fb41548c978191"}
Jul 18 02:16:21.526223 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:21.526173    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd" podStartSLOduration=1.526151553 podStartE2EDuration="1.526151553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:16:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:16:21.524060508 +0000 UTC m=+1686.963557450" watchObservedRunningTime="2026-07-18 02:16:21.526151553 +0000 UTC m=+1686.965648486"
Jul 18 02:16:22.518586 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:22.518548    2552 generic.go:356] "Generic (PLEG): container finished" podID="e03c858a-a1c2-4e7f-8dbf-74a63a7dab22" containerID="26071a0605328aee3eb29625bb2863e0631b9aba6147e4b01a0362b7939062a4" exitCode=0
Jul 18 02:16:22.519080 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:22.518631    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd" event={"ID":"e03c858a-a1c2-4e7f-8dbf-74a63a7dab22","Type":"ContainerDied","Data":"26071a0605328aee3eb29625bb2863e0631b9aba6147e4b01a0362b7939062a4"}
Jul 18 02:16:23.669229 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:23.669200    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"
Jul 18 02:16:23.723260 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:23.723231    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e03c858a-a1c2-4e7f-8dbf-74a63a7dab22-maas-api-service-ca\") pod \"e03c858a-a1c2-4e7f-8dbf-74a63a7dab22\" (UID: \"e03c858a-a1c2-4e7f-8dbf-74a63a7dab22\") "
Jul 18 02:16:23.723638 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:23.723616    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e03c858a-a1c2-4e7f-8dbf-74a63a7dab22-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e03c858a-a1c2-4e7f-8dbf-74a63a7dab22" (UID: "e03c858a-a1c2-4e7f-8dbf-74a63a7dab22"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:16:23.824130 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:23.824046    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e03c858a-a1c2-4e7f-8dbf-74a63a7dab22-maas-api-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:16:24.533611 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:24.533578    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd" event={"ID":"e03c858a-a1c2-4e7f-8dbf-74a63a7dab22","Type":"ContainerDied","Data":"578359b4731bad63339e3ca6b06d1b07871f4068ceef72fb04fb41548c978191"}
Jul 18 02:16:24.533611 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:24.533607    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"
Jul 18 02:16:24.533799 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:24.533627    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="578359b4731bad63339e3ca6b06d1b07871f4068ceef72fb04fb41548c978191"
Jul 18 02:16:30.931859 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:30.931817    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"]
Jul 18 02:16:30.932312 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:30.932083    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp" podUID="b07dd288-c6b9-4630-811c-cf89c91d99a6" containerName="maas-api" containerID="cri-o://826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6" gracePeriod=30
Jul 18 02:16:30.956502 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:30.956463    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:31.195736 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.195677    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:16:31.292935 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.292904    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b07dd288-c6b9-4630-811c-cf89c91d99a6-maas-api-tls\") pod \"b07dd288-c6b9-4630-811c-cf89c91d99a6\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") "
Jul 18 02:16:31.293140 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.292948    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbs7q\" (UniqueName: \"kubernetes.io/projected/b07dd288-c6b9-4630-811c-cf89c91d99a6-kube-api-access-rbs7q\") pod \"b07dd288-c6b9-4630-811c-cf89c91d99a6\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") "
Jul 18 02:16:31.293140 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.292996    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b07dd288-c6b9-4630-811c-cf89c91d99a6-openshift-service-ca\") pod \"b07dd288-c6b9-4630-811c-cf89c91d99a6\" (UID: \"b07dd288-c6b9-4630-811c-cf89c91d99a6\") "
Jul 18 02:16:31.293462 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.293406    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b07dd288-c6b9-4630-811c-cf89c91d99a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b07dd288-c6b9-4630-811c-cf89c91d99a6" (UID: "b07dd288-c6b9-4630-811c-cf89c91d99a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:16:31.295248 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.295225    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b07dd288-c6b9-4630-811c-cf89c91d99a6-kube-api-access-rbs7q" (OuterVolumeSpecName: "kube-api-access-rbs7q") pod "b07dd288-c6b9-4630-811c-cf89c91d99a6" (UID: "b07dd288-c6b9-4630-811c-cf89c91d99a6"). InnerVolumeSpecName "kube-api-access-rbs7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:16:31.295331 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.295226    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b07dd288-c6b9-4630-811c-cf89c91d99a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b07dd288-c6b9-4630-811c-cf89c91d99a6" (UID: "b07dd288-c6b9-4630-811c-cf89c91d99a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:16:31.394183 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.394153    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b07dd288-c6b9-4630-811c-cf89c91d99a6-maas-api-tls\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:16:31.394183 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.394180    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbs7q\" (UniqueName: \"kubernetes.io/projected/b07dd288-c6b9-4630-811c-cf89c91d99a6-kube-api-access-rbs7q\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:16:31.394183 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.394190    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b07dd288-c6b9-4630-811c-cf89c91d99a6-openshift-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:16:31.578680 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.578589    2552 generic.go:356] "Generic (PLEG): container finished" podID="b07dd288-c6b9-4630-811c-cf89c91d99a6" containerID="826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6" exitCode=0
Jul 18 02:16:31.578680 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.578642    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"
Jul 18 02:16:31.578873 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.578683    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp" event={"ID":"b07dd288-c6b9-4630-811c-cf89c91d99a6","Type":"ContainerDied","Data":"826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6"}
Jul 18 02:16:31.578873 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.578739    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp" event={"ID":"b07dd288-c6b9-4630-811c-cf89c91d99a6","Type":"ContainerDied","Data":"644690b443c4b459f41194c3399fc38e351efd8f538a40f48c97e1e0136035b3"}
Jul 18 02:16:31.578873 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.578762    2552 scope.go:117] "RemoveContainer" containerID="826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6"
Jul 18 02:16:31.588690 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.588660    2552 scope.go:117] "RemoveContainer" containerID="826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6"
Jul 18 02:16:31.588968 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:16:31.588951    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6\": container with ID starting with 826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6 not found: ID does not exist" containerID="826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6"
Jul 18 02:16:31.589015 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.588977    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6"} err="failed to get container status \"826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6\": rpc error: code = NotFound desc = could not find container \"826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6\": container with ID starting with 826a4e93edd80c5774ca7503c74b3f5d996a9d6a84044fc75a047e8f9037a3d6 not found: ID does not exist"
Jul 18 02:16:31.600847 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.600821    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"]
Jul 18 02:16:31.603378 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:31.603359    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-92aad9cd-7c69d4b67f-wcmcp"]
Jul 18 02:16:33.085975 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:33.085940    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b07dd288-c6b9-4630-811c-cf89c91d99a6" path="/var/lib/kubelet/pods/b07dd288-c6b9-4630-811c-cf89c91d99a6/volumes"
Jul 18 02:16:35.544728 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:35.544695    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"]
Jul 18 02:16:35.547903 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:35.547875    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92aad9cd-tr8jd"]
Jul 18 02:16:35.548585 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:35.548563    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:36.738317 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:36.738284    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:36.957298 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:36.957262    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:37.076366 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:37.076285    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e03c858a-a1c2-4e7f-8dbf-74a63a7dab22" path="/var/lib/kubelet/pods/e03c858a-a1c2-4e7f-8dbf-74a63a7dab22/volumes"
Jul 18 02:16:37.188522 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:37.188491    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:37.497050 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:37.497021    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:37.738947 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:37.738917    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:39.785708 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.785660    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:39.864244 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.864194    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:39.870621 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.870586    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"]
Jul 18 02:16:39.871057 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.871043    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e03c858a-a1c2-4e7f-8dbf-74a63a7dab22" containerName="revoke-keys"
Jul 18 02:16:39.871110 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.871059    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e03c858a-a1c2-4e7f-8dbf-74a63a7dab22" containerName="revoke-keys"
Jul 18 02:16:39.871110 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.871071    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b07dd288-c6b9-4630-811c-cf89c91d99a6" containerName="maas-api"
Jul 18 02:16:39.871110 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.871076    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="b07dd288-c6b9-4630-811c-cf89c91d99a6" containerName="maas-api"
Jul 18 02:16:39.871206 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.871156    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="b07dd288-c6b9-4630-811c-cf89c91d99a6" containerName="maas-api"
Jul 18 02:16:39.871206 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.871163    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e03c858a-a1c2-4e7f-8dbf-74a63a7dab22" containerName="revoke-keys"
Jul 18 02:16:39.875827 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.875808    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:39.878678 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.878659    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-ec7c9b7e-serving-cert\""
Jul 18 02:16:39.886291 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.886267    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"]
Jul 18 02:16:39.947864 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.947827    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:39.975169 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.975134    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-openshift-service-ca\") pod \"maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:39.975337 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.975198    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-maas-api-tls\") pod \"maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:39.975337 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:39.975272    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh278\" (UniqueName: \"kubernetes.io/projected/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-kube-api-access-hh278\") pod \"maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:40.075946 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.075859    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh278\" (UniqueName: \"kubernetes.io/projected/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-kube-api-access-hh278\") pod \"maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:40.075946 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.075906    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-openshift-service-ca\") pod \"maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:40.076134 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.075957    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-maas-api-tls\") pod \"maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:40.076134 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:16:40.076058    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-serving-cert: secret "maas-api-e2e-derive-ec7c9b7e-serving-cert" not found
Jul 18 02:16:40.076134 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:16:40.076130    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-maas-api-tls podName:d8814821-e1b8-4d03-a1e2-5f4b35101e8a nodeName:}" failed. No retries permitted until 2026-07-18 02:16:40.576111311 +0000 UTC m=+1706.015608221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-maas-api-tls") pod "maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr" (UID: "d8814821-e1b8-4d03-a1e2-5f4b35101e8a") : secret "maas-api-e2e-derive-ec7c9b7e-serving-cert" not found
Jul 18 02:16:40.076586 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.076566    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-openshift-service-ca\") pod \"maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:40.088862 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.088831    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh278\" (UniqueName: \"kubernetes.io/projected/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-kube-api-access-hh278\") pod \"maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:40.157865 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.157831    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:40.529083 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.529024    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:40.580790 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.580755    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-maas-api-tls\") pod \"maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:40.583409 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.583381    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-maas-api-tls\") pod \"maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:40.629738 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.629712    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:16:40.787078 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.786985    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:40.944957 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:40.944932    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"]
Jul 18 02:16:40.947019 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:16:40.946992    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8814821_e1b8_4d03_a1e2_5f4b35101e8a.slice/crio-3c8bb37ed731bbfe8c3898ce9fcdcc5bf42a16e351431205f62efdcf95ee3807 WatchSource:0}: Error finding container 3c8bb37ed731bbfe8c3898ce9fcdcc5bf42a16e351431205f62efdcf95ee3807: Status 404 returned error can't find the container with id 3c8bb37ed731bbfe8c3898ce9fcdcc5bf42a16e351431205f62efdcf95ee3807
Jul 18 02:16:41.639174 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:41.639076    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr" event={"ID":"d8814821-e1b8-4d03-a1e2-5f4b35101e8a","Type":"ContainerStarted","Data":"e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549"}
Jul 18 02:16:41.639174 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:41.639133    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr" event={"ID":"d8814821-e1b8-4d03-a1e2-5f4b35101e8a","Type":"ContainerStarted","Data":"3c8bb37ed731bbfe8c3898ce9fcdcc5bf42a16e351431205f62efdcf95ee3807"}
Jul 18 02:16:41.639460 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:41.639252    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:16:41.656549 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:41.656491    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr" podStartSLOduration=2.312909454 podStartE2EDuration="2.656472479s" totalImagesPullingTime="343.563025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:16:41.654588599 +0000 UTC m=+1707.094085550" watchObservedRunningTime="2026-07-18 02:16:41.656472479 +0000 UTC m=+1707.095969455"
Jul 18 02:16:47.652164 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:16:47.652134    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:17:04.653905 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:04.653873    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"]
Jul 18 02:17:04.657411 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:04.657395    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"
Jul 18 02:17:04.659621 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:04.659601    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vpdlz\""
Jul 18 02:17:04.672966 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:04.669904    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"]
Jul 18 02:17:04.802919 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:04.802881    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7\" (UID: \"a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"
Jul 18 02:17:04.904367 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:04.904276    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7\" (UID: \"a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"
Jul 18 02:17:04.904967 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:04.904946    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7\" (UID: \"a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"
Jul 18 02:17:04.970101 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:04.970058    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"
Jul 18 02:17:05.119649 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:05.119622    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"]
Jul 18 02:17:05.121720 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:17:05.121692    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda388c7fe_c8f5_4449_bb8f_6cd9a1b0d662.slice/crio-ce57ea3fe8ec1672f2048f6ca9b3fe47dde94ce0a8150b6695262e36904186b5 WatchSource:0}: Error finding container ce57ea3fe8ec1672f2048f6ca9b3fe47dde94ce0a8150b6695262e36904186b5: Status 404 returned error can't find the container with id ce57ea3fe8ec1672f2048f6ca9b3fe47dde94ce0a8150b6695262e36904186b5
Jul 18 02:17:05.783134 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:05.783100    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7" event={"ID":"a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662","Type":"ContainerStarted","Data":"fd7ff9a3f3821b280ddde131b7dd556cbda2341ecb1e3340d6b983c07aab688a"}
Jul 18 02:17:05.783536 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:05.783152    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7" event={"ID":"a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662","Type":"ContainerStarted","Data":"ce57ea3fe8ec1672f2048f6ca9b3fe47dde94ce0a8150b6695262e36904186b5"}
Jul 18 02:17:05.798470 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:05.798406    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7" podStartSLOduration=1.798390369 podStartE2EDuration="1.798390369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:17:05.796213541 +0000 UTC m=+1731.235710486" watchObservedRunningTime="2026-07-18 02:17:05.798390369 +0000 UTC m=+1731.237887300"
Jul 18 02:17:06.790862 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:06.790824    2552 generic.go:356] "Generic (PLEG): container finished" podID="a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662" containerID="fd7ff9a3f3821b280ddde131b7dd556cbda2341ecb1e3340d6b983c07aab688a" exitCode=0
Jul 18 02:17:06.791325 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:06.790921    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7" event={"ID":"a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662","Type":"ContainerDied","Data":"fd7ff9a3f3821b280ddde131b7dd556cbda2341ecb1e3340d6b983c07aab688a"}
Jul 18 02:17:07.935696 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:07.935671    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"
Jul 18 02:17:08.037029 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:08.036977    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662-maas-api-service-ca\") pod \"a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662\" (UID: \"a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662\") "
Jul 18 02:17:08.037363 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:08.037339    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662" (UID: "a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:17:08.137794 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:08.137704    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662-maas-api-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:17:08.803749 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:08.803718    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"
Jul 18 02:17:08.803749 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:08.803726    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7" event={"ID":"a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662","Type":"ContainerDied","Data":"ce57ea3fe8ec1672f2048f6ca9b3fe47dde94ce0a8150b6695262e36904186b5"}
Jul 18 02:17:08.803971 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:08.803777    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce57ea3fe8ec1672f2048f6ca9b3fe47dde94ce0a8150b6695262e36904186b5"
Jul 18 02:17:15.059439 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.059382    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"]
Jul 18 02:17:15.060279 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.060191    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr" podUID="d8814821-e1b8-4d03-a1e2-5f4b35101e8a" containerName="maas-api" containerID="cri-o://e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549" gracePeriod=30
Jul 18 02:17:15.076922 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.076863    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:15.316652 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.316582    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:17:15.394226 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.394191    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-openshift-service-ca\") pod \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") "
Jul 18 02:17:15.394401 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.394251    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-maas-api-tls\") pod \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") "
Jul 18 02:17:15.394479 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.394455    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hh278\" (UniqueName: \"kubernetes.io/projected/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-kube-api-access-hh278\") pod \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\" (UID: \"d8814821-e1b8-4d03-a1e2-5f4b35101e8a\") "
Jul 18 02:17:15.394602 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.394577    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d8814821-e1b8-4d03-a1e2-5f4b35101e8a" (UID: "d8814821-e1b8-4d03-a1e2-5f4b35101e8a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:17:15.394780 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.394756    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-openshift-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:17:15.396626 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.396602    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d8814821-e1b8-4d03-a1e2-5f4b35101e8a" (UID: "d8814821-e1b8-4d03-a1e2-5f4b35101e8a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:17:15.396711 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.396619    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-kube-api-access-hh278" (OuterVolumeSpecName: "kube-api-access-hh278") pod "d8814821-e1b8-4d03-a1e2-5f4b35101e8a" (UID: "d8814821-e1b8-4d03-a1e2-5f4b35101e8a"). InnerVolumeSpecName "kube-api-access-hh278". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:17:15.495334 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.495280    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh278\" (UniqueName: \"kubernetes.io/projected/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-kube-api-access-hh278\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:17:15.495334 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.495328    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8814821-e1b8-4d03-a1e2-5f4b35101e8a-maas-api-tls\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:17:15.850084 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.850050    2552 generic.go:356] "Generic (PLEG): container finished" podID="d8814821-e1b8-4d03-a1e2-5f4b35101e8a" containerID="e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549" exitCode=0
Jul 18 02:17:15.850315 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.850101    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"
Jul 18 02:17:15.850315 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.850145    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr" event={"ID":"d8814821-e1b8-4d03-a1e2-5f4b35101e8a","Type":"ContainerDied","Data":"e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549"}
Jul 18 02:17:15.850315 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.850201    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr" event={"ID":"d8814821-e1b8-4d03-a1e2-5f4b35101e8a","Type":"ContainerDied","Data":"3c8bb37ed731bbfe8c3898ce9fcdcc5bf42a16e351431205f62efdcf95ee3807"}
Jul 18 02:17:15.850315 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.850226    2552 scope.go:117] "RemoveContainer" containerID="e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549"
Jul 18 02:17:15.860244 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.860225    2552 scope.go:117] "RemoveContainer" containerID="e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549"
Jul 18 02:17:15.860540 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:17:15.860517    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549\": container with ID starting with e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549 not found: ID does not exist" containerID="e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549"
Jul 18 02:17:15.860620 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.860551    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549"} err="failed to get container status \"e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549\": rpc error: code = NotFound desc = could not find container \"e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549\": container with ID starting with e112d9cf8bad73d8f319b2f4d010fba92e633d626a22aafcadff515ec8692549 not found: ID does not exist"
Jul 18 02:17:15.871678 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.871651    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"]
Jul 18 02:17:15.875194 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:15.875172    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ec7c9b7e-59f455595d-qn9fr"]
Jul 18 02:17:17.076441 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:17.076392    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8814821-e1b8-4d03-a1e2-5f4b35101e8a" path="/var/lib/kubelet/pods/d8814821-e1b8-4d03-a1e2-5f4b35101e8a/volumes"
Jul 18 02:17:19.730331 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:19.730288    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"]
Jul 18 02:17:19.736908 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:19.736871    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:19.738531 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:19.738511    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ec7c9b7e-zjrq7"]
Jul 18 02:17:19.892829 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:19.892792    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:20.133925 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:20.133825    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:20.618975 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:20.618932    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:20.716970 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:20.716940    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:20.770333 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:20.770304    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:21.076043 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:21.076010    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662" path="/var/lib/kubelet/pods/a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662/volumes"
Jul 18 02:17:22.324395 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:22.324361    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:28.788719 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:28.788646    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:28.893658 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:28.893622    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:28.989237 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:28.989207    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:29.087386 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:29.087297    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:29.540646 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:29.540615    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:34.500561 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:34.500529    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:34.602830 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:34.602750    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:55.560780 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:55.560745    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:56.873233 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:56.873204    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:17:56.974804 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:17:56.974771    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:01.982045 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:01.982017    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:02.089629 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:02.089601    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:02.187572 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:02.187533    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:15.161747 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:15.161719    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:18:15.168223 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:15.168199    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:18:18.077012 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:18.076971    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:23.689293 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:23.689256    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:23.799996 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:23.799964    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:23.890627 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:23.890596    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:24.506468 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:24.506412    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:24.606643 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:24.606610    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:34.445870 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:34.445833    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:35.286373 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:35.286339    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:41.494431 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:41.494369    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:41.600819 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:41.600789    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:41.697191 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:41.697158    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:41.800480 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:41.800370    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:42.705263 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:42.705228    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:42.806037 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:42.805999    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:44.564998 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:44.564961    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:51.052487 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:51.052445    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:51.153598 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:51.153567    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:51.737069 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:51.737029    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:57.884736 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:57.884704    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:57.983172 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:57.983141    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:58.146911 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:58.146883    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:58.466275 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:58.466194    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:18:58.741674 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:18:58.741590    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:01.341293 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.341259    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:01.417761 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.417728    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"]
Jul 18 02:19:01.418465 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.418414    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8814821-e1b8-4d03-a1e2-5f4b35101e8a" containerName="maas-api"
Jul 18 02:19:01.418523 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.418467    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8814821-e1b8-4d03-a1e2-5f4b35101e8a" containerName="maas-api"
Jul 18 02:19:01.418523 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.418495    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662" containerName="revoke-keys"
Jul 18 02:19:01.418523 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.418504    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662" containerName="revoke-keys"
Jul 18 02:19:01.418627 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.418615    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8814821-e1b8-4d03-a1e2-5f4b35101e8a" containerName="maas-api"
Jul 18 02:19:01.418666 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.418636    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a388c7fe-c8f5-4449-bb8f-6cd9a1b0d662" containerName="revoke-keys"
Jul 18 02:19:01.422323 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.422301    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:01.424773 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.424750    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-084f79d4-serving-cert\""
Jul 18 02:19:01.434620 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.434586    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"]
Jul 18 02:19:01.436330 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.436308    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:01.456576 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.456548    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40e4d64d-bc25-429f-93ea-a33e573c5b60-maas-api-tls\") pod \"maas-api-e2e-mt-084f79d4-5df6749975-6zkrd\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:01.456746 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.456591    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40e4d64d-bc25-429f-93ea-a33e573c5b60-openshift-service-ca\") pod \"maas-api-e2e-mt-084f79d4-5df6749975-6zkrd\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:01.456746 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.456619    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsxtn\" (UniqueName: \"kubernetes.io/projected/40e4d64d-bc25-429f-93ea-a33e573c5b60-kube-api-access-dsxtn\") pod \"maas-api-e2e-mt-084f79d4-5df6749975-6zkrd\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:01.514145 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.514110    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:01.557485 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.557413    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40e4d64d-bc25-429f-93ea-a33e573c5b60-maas-api-tls\") pod \"maas-api-e2e-mt-084f79d4-5df6749975-6zkrd\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:01.557682 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.557510    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40e4d64d-bc25-429f-93ea-a33e573c5b60-openshift-service-ca\") pod \"maas-api-e2e-mt-084f79d4-5df6749975-6zkrd\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:01.557682 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.557565    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsxtn\" (UniqueName: \"kubernetes.io/projected/40e4d64d-bc25-429f-93ea-a33e573c5b60-kube-api-access-dsxtn\") pod \"maas-api-e2e-mt-084f79d4-5df6749975-6zkrd\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:01.557682 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:19:01.557578    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-serving-cert: secret "maas-api-e2e-mt-084f79d4-serving-cert" not found
Jul 18 02:19:01.557682 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:19:01.557640    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40e4d64d-bc25-429f-93ea-a33e573c5b60-maas-api-tls podName:40e4d64d-bc25-429f-93ea-a33e573c5b60 nodeName:}" failed. No retries permitted until 2026-07-18 02:19:02.057620156 +0000 UTC m=+1847.497117085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/40e4d64d-bc25-429f-93ea-a33e573c5b60-maas-api-tls") pod "maas-api-e2e-mt-084f79d4-5df6749975-6zkrd" (UID: "40e4d64d-bc25-429f-93ea-a33e573c5b60") : secret "maas-api-e2e-mt-084f79d4-serving-cert" not found
Jul 18 02:19:01.558168 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.558147    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40e4d64d-bc25-429f-93ea-a33e573c5b60-openshift-service-ca\") pod \"maas-api-e2e-mt-084f79d4-5df6749975-6zkrd\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:01.569372 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.569345    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsxtn\" (UniqueName: \"kubernetes.io/projected/40e4d64d-bc25-429f-93ea-a33e573c5b60-kube-api-access-dsxtn\") pod \"maas-api-e2e-mt-084f79d4-5df6749975-6zkrd\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:01.663345 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.663315    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:01.999885 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:01.999800    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:02.062594 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:02.062563    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40e4d64d-bc25-429f-93ea-a33e573c5b60-maas-api-tls\") pod \"maas-api-e2e-mt-084f79d4-5df6749975-6zkrd\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:02.065095 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:02.065072    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40e4d64d-bc25-429f-93ea-a33e573c5b60-maas-api-tls\") pod \"maas-api-e2e-mt-084f79d4-5df6749975-6zkrd\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:02.104514 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:02.104471    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:02.333560 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:02.333461    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:02.480554 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:02.480529    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"]
Jul 18 02:19:02.482083 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:19:02.482058    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e4d64d_bc25_429f_93ea_a33e573c5b60.slice/crio-c986675df17cfad8e8c49e07a416e4f7836414afdfe15d18f7098dc1f3b26369 WatchSource:0}: Error finding container c986675df17cfad8e8c49e07a416e4f7836414afdfe15d18f7098dc1f3b26369: Status 404 returned error can't find the container with id c986675df17cfad8e8c49e07a416e4f7836414afdfe15d18f7098dc1f3b26369
Jul 18 02:19:03.477963 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:03.477925    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd" event={"ID":"40e4d64d-bc25-429f-93ea-a33e573c5b60","Type":"ContainerStarted","Data":"2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c"}
Jul 18 02:19:03.478154 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:03.477989    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd" event={"ID":"40e4d64d-bc25-429f-93ea-a33e573c5b60","Type":"ContainerStarted","Data":"c986675df17cfad8e8c49e07a416e4f7836414afdfe15d18f7098dc1f3b26369"}
Jul 18 02:19:03.478154 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:03.478020    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:03.494729 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:03.494682    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd" podStartSLOduration=2.098745044 podStartE2EDuration="2.494665019s" totalImagesPullingTime="395.919975ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:19:03.492666756 +0000 UTC m=+1848.932163687" watchObservedRunningTime="2026-07-18 02:19:03.494665019 +0000 UTC m=+1848.934161951"
Jul 18 02:19:05.839740 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:05.839708    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:06.583668 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:06.583635    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:06.676499 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:06.676463    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:06.970361 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:06.970330    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:09.488946 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:09.488911    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:12.907852 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:12.907808    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:13.009152 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:13.009113    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:13.270647 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:13.270561    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"]
Jul 18 02:19:13.274579 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:13.274560    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"
Jul 18 02:19:13.276899 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:13.276879    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vpdlz\""
Jul 18 02:19:13.281484 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:13.281460    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"]
Jul 18 02:19:13.369806 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:13.369776    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e904238b-f851-40d2-bffa-9fcb022a67e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp\" (UID: \"e904238b-f851-40d2-bffa-9fcb022a67e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"
Jul 18 02:19:13.470252 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:13.470209    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e904238b-f851-40d2-bffa-9fcb022a67e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp\" (UID: \"e904238b-f851-40d2-bffa-9fcb022a67e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"
Jul 18 02:19:13.470856 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:13.470836    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e904238b-f851-40d2-bffa-9fcb022a67e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp\" (UID: \"e904238b-f851-40d2-bffa-9fcb022a67e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"
Jul 18 02:19:13.587299 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:13.587216    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"
Jul 18 02:19:13.747030 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:13.747001    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"]
Jul 18 02:19:13.748618 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:19:13.748593    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode904238b_f851_40d2_bffa_9fcb022a67e2.slice/crio-a4b4da06b99deb5e4e8c3b2c19a4d5bb56933cd41b1a5077fd09420686cf7f83 WatchSource:0}: Error finding container a4b4da06b99deb5e4e8c3b2c19a4d5bb56933cd41b1a5077fd09420686cf7f83: Status 404 returned error can't find the container with id a4b4da06b99deb5e4e8c3b2c19a4d5bb56933cd41b1a5077fd09420686cf7f83
Jul 18 02:19:14.551677 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:14.551643    2552 generic.go:356] "Generic (PLEG): container finished" podID="e904238b-f851-40d2-bffa-9fcb022a67e2" containerID="402e19c5bd03b891b044c0cbd3d8c3e8ce12e96cf82bd404c367c1857b6c33a4" exitCode=0
Jul 18 02:19:14.552049 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:14.551690    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp" event={"ID":"e904238b-f851-40d2-bffa-9fcb022a67e2","Type":"ContainerDied","Data":"402e19c5bd03b891b044c0cbd3d8c3e8ce12e96cf82bd404c367c1857b6c33a4"}
Jul 18 02:19:14.552049 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:14.551731    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp" event={"ID":"e904238b-f851-40d2-bffa-9fcb022a67e2","Type":"ContainerStarted","Data":"a4b4da06b99deb5e4e8c3b2c19a4d5bb56933cd41b1a5077fd09420686cf7f83"}
Jul 18 02:19:15.705856 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:15.705827    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"
Jul 18 02:19:15.790800 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:15.790767    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e904238b-f851-40d2-bffa-9fcb022a67e2-maas-api-service-ca\") pod \"e904238b-f851-40d2-bffa-9fcb022a67e2\" (UID: \"e904238b-f851-40d2-bffa-9fcb022a67e2\") "
Jul 18 02:19:15.791150 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:15.791128    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e904238b-f851-40d2-bffa-9fcb022a67e2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e904238b-f851-40d2-bffa-9fcb022a67e2" (UID: "e904238b-f851-40d2-bffa-9fcb022a67e2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:19:15.892381 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:15.892355    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e904238b-f851-40d2-bffa-9fcb022a67e2-maas-api-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:19:16.565345 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:16.565313    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"
Jul 18 02:19:16.565345 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:16.565324    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp" event={"ID":"e904238b-f851-40d2-bffa-9fcb022a67e2","Type":"ContainerDied","Data":"a4b4da06b99deb5e4e8c3b2c19a4d5bb56933cd41b1a5077fd09420686cf7f83"}
Jul 18 02:19:16.565599 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:16.565393    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4b4da06b99deb5e4e8c3b2c19a4d5bb56933cd41b1a5077fd09420686cf7f83"
Jul 18 02:19:23.792571 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:23.792538    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"]
Jul 18 02:19:23.792957 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:23.792755    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd" podUID="40e4d64d-bc25-429f-93ea-a33e573c5b60" containerName="maas-api" containerID="cri-o://2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c" gracePeriod=30
Jul 18 02:19:23.819671 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:23.819647    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:24.052278 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.052211    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:24.170619 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.170580    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40e4d64d-bc25-429f-93ea-a33e573c5b60-maas-api-tls\") pod \"40e4d64d-bc25-429f-93ea-a33e573c5b60\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") "
Jul 18 02:19:24.170619 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.170623    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsxtn\" (UniqueName: \"kubernetes.io/projected/40e4d64d-bc25-429f-93ea-a33e573c5b60-kube-api-access-dsxtn\") pod \"40e4d64d-bc25-429f-93ea-a33e573c5b60\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") "
Jul 18 02:19:24.170849 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.170671    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40e4d64d-bc25-429f-93ea-a33e573c5b60-openshift-service-ca\") pod \"40e4d64d-bc25-429f-93ea-a33e573c5b60\" (UID: \"40e4d64d-bc25-429f-93ea-a33e573c5b60\") "
Jul 18 02:19:24.171087 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.171061    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40e4d64d-bc25-429f-93ea-a33e573c5b60-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40e4d64d-bc25-429f-93ea-a33e573c5b60" (UID: "40e4d64d-bc25-429f-93ea-a33e573c5b60"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:19:24.173003 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.172982    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40e4d64d-bc25-429f-93ea-a33e573c5b60-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40e4d64d-bc25-429f-93ea-a33e573c5b60" (UID: "40e4d64d-bc25-429f-93ea-a33e573c5b60"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:19:24.173073 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.172995    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e4d64d-bc25-429f-93ea-a33e573c5b60-kube-api-access-dsxtn" (OuterVolumeSpecName: "kube-api-access-dsxtn") pod "40e4d64d-bc25-429f-93ea-a33e573c5b60" (UID: "40e4d64d-bc25-429f-93ea-a33e573c5b60"). InnerVolumeSpecName "kube-api-access-dsxtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:19:24.271449 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.271388    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40e4d64d-bc25-429f-93ea-a33e573c5b60-maas-api-tls\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:19:24.271449 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.271418    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsxtn\" (UniqueName: \"kubernetes.io/projected/40e4d64d-bc25-429f-93ea-a33e573c5b60-kube-api-access-dsxtn\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:19:24.271666 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.271459    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40e4d64d-bc25-429f-93ea-a33e573c5b60-openshift-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:19:24.615248 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.615211    2552 generic.go:356] "Generic (PLEG): container finished" podID="40e4d64d-bc25-429f-93ea-a33e573c5b60" containerID="2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c" exitCode=0
Jul 18 02:19:24.615463 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.615258    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd" event={"ID":"40e4d64d-bc25-429f-93ea-a33e573c5b60","Type":"ContainerDied","Data":"2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c"}
Jul 18 02:19:24.615463 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.615330    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd" event={"ID":"40e4d64d-bc25-429f-93ea-a33e573c5b60","Type":"ContainerDied","Data":"c986675df17cfad8e8c49e07a416e4f7836414afdfe15d18f7098dc1f3b26369"}
Jul 18 02:19:24.615463 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.615268    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"
Jul 18 02:19:24.615463 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.615361    2552 scope.go:117] "RemoveContainer" containerID="2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c"
Jul 18 02:19:24.626242 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.626223    2552 scope.go:117] "RemoveContainer" containerID="2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c"
Jul 18 02:19:24.626534 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:19:24.626515    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c\": container with ID starting with 2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c not found: ID does not exist" containerID="2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c"
Jul 18 02:19:24.626605 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.626568    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c"} err="failed to get container status \"2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c\": rpc error: code = NotFound desc = could not find container \"2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c\": container with ID starting with 2229303e46073aa62f161adcddd3f2bbce637e09df66cff4617525435f5fd90c not found: ID does not exist"
Jul 18 02:19:24.636955 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.636931    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"]
Jul 18 02:19:24.639353 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:24.639331    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-084f79d4-5df6749975-6zkrd"]
Jul 18 02:19:25.076056 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:25.076026    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40e4d64d-bc25-429f-93ea-a33e573c5b60" path="/var/lib/kubelet/pods/40e4d64d-bc25-429f-93ea-a33e573c5b60/volumes"
Jul 18 02:19:28.353342 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:28.353297    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"]
Jul 18 02:19:28.357473 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:28.357446    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:28.359234 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:28.359213    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-084f79d4-c8hnp"]
Jul 18 02:19:28.734792 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:28.734761    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:29.076637 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:29.076544    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e904238b-f851-40d2-bffa-9fcb022a67e2" path="/var/lib/kubelet/pods/e904238b-f851-40d2-bffa-9fcb022a67e2/volumes"
Jul 18 02:19:34.828569 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:34.828539    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:34.898655 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:34.898631    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:35.108229 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:35.108108    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:35.350096 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:35.350058    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:35.470732 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:35.470685    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:35.564486 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:35.564441    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:35.673813 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:35.673735    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:41.217995 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:41.217959    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:41.325835 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:41.325803    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:41.418445 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:41.418386    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:41.572442 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:41.572331    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:47.260091 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:47.260057    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:47.364178 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:47.364143    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:54.162897 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:54.162859    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:54.448814 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:54.448699    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:55.273475 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:55.273435    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:57.308955 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.308918    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:57.391480 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.391447    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:57.400780 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.400743    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"]
Jul 18 02:19:57.401393 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.401373    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e4d64d-bc25-429f-93ea-a33e573c5b60" containerName="maas-api"
Jul 18 02:19:57.401473 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.401398    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e4d64d-bc25-429f-93ea-a33e573c5b60" containerName="maas-api"
Jul 18 02:19:57.401473 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.401465    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e904238b-f851-40d2-bffa-9fcb022a67e2" containerName="revoke-keys"
Jul 18 02:19:57.401541 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.401475    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e904238b-f851-40d2-bffa-9fcb022a67e2" containerName="revoke-keys"
Jul 18 02:19:57.401617 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.401604    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e904238b-f851-40d2-bffa-9fcb022a67e2" containerName="revoke-keys"
Jul 18 02:19:57.401656 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.401623    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e4d64d-bc25-429f-93ea-a33e573c5b60" containerName="maas-api"
Jul 18 02:19:57.405288 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.405271    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:57.407781 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.407761    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0cc0a379-serving-cert\""
Jul 18 02:19:57.414975 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.414952    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"]
Jul 18 02:19:57.457834 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.457800    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:57.468051 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.468014    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5jlp\" (UniqueName: \"kubernetes.io/projected/3209c53d-801c-45d0-b917-142bd3f7533f-kube-api-access-m5jlp\") pod \"maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:57.468213 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.468068    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3209c53d-801c-45d0-b917-142bd3f7533f-maas-api-tls\") pod \"maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:57.468213 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.468192    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3209c53d-801c-45d0-b917-142bd3f7533f-openshift-service-ca\") pod \"maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:57.569361 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.569265    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5jlp\" (UniqueName: \"kubernetes.io/projected/3209c53d-801c-45d0-b917-142bd3f7533f-kube-api-access-m5jlp\") pod \"maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:57.569361 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.569309    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3209c53d-801c-45d0-b917-142bd3f7533f-maas-api-tls\") pod \"maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:57.569361 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.569354    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3209c53d-801c-45d0-b917-142bd3f7533f-openshift-service-ca\") pod \"maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:57.570025 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.570006    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3209c53d-801c-45d0-b917-142bd3f7533f-openshift-service-ca\") pod \"maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:57.571988 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.571964    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3209c53d-801c-45d0-b917-142bd3f7533f-maas-api-tls\") pod \"maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:57.579454 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.579414    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5jlp\" (UniqueName: \"kubernetes.io/projected/3209c53d-801c-45d0-b917-142bd3f7533f-kube-api-access-m5jlp\") pod \"maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:57.612559 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.612529    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:57.719036 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.718994    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:57.871068 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.870990    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"]
Jul 18 02:19:57.876496 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:19:57.876462    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3209c53d_801c_45d0_b917_142bd3f7533f.slice/crio-d5afd3190afb6adb916922ea768da1a31e1e94f9f6407807d5460a2408fd0f83 WatchSource:0}: Error finding container d5afd3190afb6adb916922ea768da1a31e1e94f9f6407807d5460a2408fd0f83: Status 404 returned error can't find the container with id d5afd3190afb6adb916922ea768da1a31e1e94f9f6407807d5460a2408fd0f83
Jul 18 02:19:57.878362 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.878340    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:19:57.988378 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:57.988342    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:58.087646 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:58.087609    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:19:58.828785 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:58.828752    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8" event={"ID":"3209c53d-801c-45d0-b917-142bd3f7533f","Type":"ContainerStarted","Data":"e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8"}
Jul 18 02:19:58.829194 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:58.828804    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8" event={"ID":"3209c53d-801c-45d0-b917-142bd3f7533f","Type":"ContainerStarted","Data":"d5afd3190afb6adb916922ea768da1a31e1e94f9f6407807d5460a2408fd0f83"}
Jul 18 02:19:58.829194 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:58.828837    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:19:58.847334 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:19:58.847281    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8" podStartSLOduration=1.465090273 podStartE2EDuration="1.847266345s" totalImagesPullingTime="382.176072ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:19:58.845586494 +0000 UTC m=+1904.285083427" watchObservedRunningTime="2026-07-18 02:19:58.847266345 +0000 UTC m=+1904.286763277"
Jul 18 02:20:01.795232 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:01.795199    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:04.841512 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:04.841484    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:20:10.651660 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:10.651628    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:35.277634 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:35.277592    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"]
Jul 18 02:20:35.281347 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:35.281328    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"
Jul 18 02:20:35.283886 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:35.283859    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vpdlz\""
Jul 18 02:20:35.288477 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:35.288449    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"]
Jul 18 02:20:35.426555 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:35.426521    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f49cc4-d9db-4c6f-8f15-0ec1e824233c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw\" (UID: \"98f49cc4-d9db-4c6f-8f15-0ec1e824233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"
Jul 18 02:20:35.527186 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:35.527151    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f49cc4-d9db-4c6f-8f15-0ec1e824233c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw\" (UID: \"98f49cc4-d9db-4c6f-8f15-0ec1e824233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"
Jul 18 02:20:35.527891 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:35.527842    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f49cc4-d9db-4c6f-8f15-0ec1e824233c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw\" (UID: \"98f49cc4-d9db-4c6f-8f15-0ec1e824233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"
Jul 18 02:20:35.592818 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:35.592781    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"
Jul 18 02:20:35.742150 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:35.742122    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"]
Jul 18 02:20:35.744684 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:20:35.744657    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f49cc4_d9db_4c6f_8f15_0ec1e824233c.slice/crio-3e2f82dbebcb0a979a470ddba23b0db3398ab90d06ff4eefb81a6f8a3aff74d8 WatchSource:0}: Error finding container 3e2f82dbebcb0a979a470ddba23b0db3398ab90d06ff4eefb81a6f8a3aff74d8: Status 404 returned error can't find the container with id 3e2f82dbebcb0a979a470ddba23b0db3398ab90d06ff4eefb81a6f8a3aff74d8
Jul 18 02:20:36.053986 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:36.053888    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw" event={"ID":"98f49cc4-d9db-4c6f-8f15-0ec1e824233c","Type":"ContainerStarted","Data":"bb5b82d0f1f381d6502e90bb9fb3ac2f4f47a224be02cd8cf02e88d767293dd3"}
Jul 18 02:20:36.053986 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:36.053944    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw" event={"ID":"98f49cc4-d9db-4c6f-8f15-0ec1e824233c","Type":"ContainerStarted","Data":"3e2f82dbebcb0a979a470ddba23b0db3398ab90d06ff4eefb81a6f8a3aff74d8"}
Jul 18 02:20:36.070398 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:36.070350    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw" podStartSLOduration=1.070334465 podStartE2EDuration="1.070334465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:20:36.068480395 +0000 UTC m=+1941.507977355" watchObservedRunningTime="2026-07-18 02:20:36.070334465 +0000 UTC m=+1941.509831397"
Jul 18 02:20:37.061748 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:37.061716    2552 generic.go:356] "Generic (PLEG): container finished" podID="98f49cc4-d9db-4c6f-8f15-0ec1e824233c" containerID="bb5b82d0f1f381d6502e90bb9fb3ac2f4f47a224be02cd8cf02e88d767293dd3" exitCode=0
Jul 18 02:20:37.062147 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:37.061805    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw" event={"ID":"98f49cc4-d9db-4c6f-8f15-0ec1e824233c","Type":"ContainerDied","Data":"bb5b82d0f1f381d6502e90bb9fb3ac2f4f47a224be02cd8cf02e88d767293dd3"}
Jul 18 02:20:38.210796 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:38.210766    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"
Jul 18 02:20:38.250689 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:38.250655    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f49cc4-d9db-4c6f-8f15-0ec1e824233c-maas-api-service-ca\") pod \"98f49cc4-d9db-4c6f-8f15-0ec1e824233c\" (UID: \"98f49cc4-d9db-4c6f-8f15-0ec1e824233c\") "
Jul 18 02:20:38.251030 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:38.251004    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98f49cc4-d9db-4c6f-8f15-0ec1e824233c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "98f49cc4-d9db-4c6f-8f15-0ec1e824233c" (UID: "98f49cc4-d9db-4c6f-8f15-0ec1e824233c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:20:38.352063 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:38.351974    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f49cc4-d9db-4c6f-8f15-0ec1e824233c-maas-api-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:20:39.075712 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:39.075687    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"
Jul 18 02:20:39.076926 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:39.076903    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw" event={"ID":"98f49cc4-d9db-4c6f-8f15-0ec1e824233c","Type":"ContainerDied","Data":"3e2f82dbebcb0a979a470ddba23b0db3398ab90d06ff4eefb81a6f8a3aff74d8"}
Jul 18 02:20:39.076988 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:39.076944    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e2f82dbebcb0a979a470ddba23b0db3398ab90d06ff4eefb81a6f8a3aff74d8"
Jul 18 02:20:50.814656 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:50.814615    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"]
Jul 18 02:20:50.815265 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:50.814890    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8" podUID="3209c53d-801c-45d0-b917-142bd3f7533f" containerName="maas-api" containerID="cri-o://e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8" gracePeriod=30
Jul 18 02:20:50.829565 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:50.829522    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:51.079015 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.078950    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:20:51.147970 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.147930    2552 generic.go:356] "Generic (PLEG): container finished" podID="3209c53d-801c-45d0-b917-142bd3f7533f" containerID="e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8" exitCode=0
Jul 18 02:20:51.148136 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.147977    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"
Jul 18 02:20:51.148136 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.148013    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8" event={"ID":"3209c53d-801c-45d0-b917-142bd3f7533f","Type":"ContainerDied","Data":"e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8"}
Jul 18 02:20:51.148136 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.148070    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8" event={"ID":"3209c53d-801c-45d0-b917-142bd3f7533f","Type":"ContainerDied","Data":"d5afd3190afb6adb916922ea768da1a31e1e94f9f6407807d5460a2408fd0f83"}
Jul 18 02:20:51.148136 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.148097    2552 scope.go:117] "RemoveContainer" containerID="e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8"
Jul 18 02:20:51.157548 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.157522    2552 scope.go:117] "RemoveContainer" containerID="e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8"
Jul 18 02:20:51.157830 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:20:51.157810    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8\": container with ID starting with e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8 not found: ID does not exist" containerID="e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8"
Jul 18 02:20:51.157904 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.157847    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8"} err="failed to get container status \"e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8\": rpc error: code = NotFound desc = could not find container \"e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8\": container with ID starting with e31ea8add2afe6f3647e1bb8412f25b4da6af09684960814977b4ef0a34711e8 not found: ID does not exist"
Jul 18 02:20:51.172253 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.172220    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5jlp\" (UniqueName: \"kubernetes.io/projected/3209c53d-801c-45d0-b917-142bd3f7533f-kube-api-access-m5jlp\") pod \"3209c53d-801c-45d0-b917-142bd3f7533f\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") "
Jul 18 02:20:51.172392 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.172287    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3209c53d-801c-45d0-b917-142bd3f7533f-maas-api-tls\") pod \"3209c53d-801c-45d0-b917-142bd3f7533f\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") "
Jul 18 02:20:51.172392 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.172325    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3209c53d-801c-45d0-b917-142bd3f7533f-openshift-service-ca\") pod \"3209c53d-801c-45d0-b917-142bd3f7533f\" (UID: \"3209c53d-801c-45d0-b917-142bd3f7533f\") "
Jul 18 02:20:51.172756 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.172734    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3209c53d-801c-45d0-b917-142bd3f7533f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3209c53d-801c-45d0-b917-142bd3f7533f" (UID: "3209c53d-801c-45d0-b917-142bd3f7533f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:20:51.174641 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.174618    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3209c53d-801c-45d0-b917-142bd3f7533f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3209c53d-801c-45d0-b917-142bd3f7533f" (UID: "3209c53d-801c-45d0-b917-142bd3f7533f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:20:51.174641 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.174623    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3209c53d-801c-45d0-b917-142bd3f7533f-kube-api-access-m5jlp" (OuterVolumeSpecName: "kube-api-access-m5jlp") pod "3209c53d-801c-45d0-b917-142bd3f7533f" (UID: "3209c53d-801c-45d0-b917-142bd3f7533f"). InnerVolumeSpecName "kube-api-access-m5jlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:20:51.273855 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.273817    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5jlp\" (UniqueName: \"kubernetes.io/projected/3209c53d-801c-45d0-b917-142bd3f7533f-kube-api-access-m5jlp\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:20:51.273855 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.273849    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3209c53d-801c-45d0-b917-142bd3f7533f-maas-api-tls\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:20:51.273855 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.273858    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3209c53d-801c-45d0-b917-142bd3f7533f-openshift-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:20:51.464365 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:20:51.464332    2552 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3209c53d_801c_45d0_b917_142bd3f7533f.slice/crio-d5afd3190afb6adb916922ea768da1a31e1e94f9f6407807d5460a2408fd0f83\": RecentStats: unable to find data in memory cache]"
Jul 18 02:20:51.488914 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.488876    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"]
Jul 18 02:20:51.494070 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:51.494033    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc0a379-5cfc8bb99-2rjf8"]
Jul 18 02:20:53.076906 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:53.076869    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3209c53d-801c-45d0-b917-142bd3f7533f" path="/var/lib/kubelet/pods/3209c53d-801c-45d0-b917-142bd3f7533f/volumes"
Jul 18 02:20:55.355566 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:55.355522    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:20:55.365006 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:55.364980    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"]
Jul 18 02:20:55.371470 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:55.371448    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0cc0a379-jbjcw"]
Jul 18 02:20:57.075782 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:57.075746    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98f49cc4-d9db-4c6f-8f15-0ec1e824233c" path="/var/lib/kubelet/pods/98f49cc4-d9db-4c6f-8f15-0ec1e824233c/volumes"
Jul 18 02:20:57.668441 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:20:57.668400    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:02.692393 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:02.692358    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:15.307912 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:15.307872    2552 scope.go:117] "RemoveContainer" containerID="89b5f033f839c43f54ec82741ee7e88a626fc941921c0efe9d4f30c709e2af6c"
Jul 18 02:21:15.316966 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:15.316948    2552 scope.go:117] "RemoveContainer" containerID="4febe29a3dcb238c37e9f713e103a1d6d029c3f13447e057a25d673379d58537"
Jul 18 02:21:39.669469 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:39.669409    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:39.813413 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:39.813376    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:45.246253 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:45.246220    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:45.346924 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:45.346884    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:45.540584 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:45.540502    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:45.775899 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:45.775869    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:46.878049 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:46.878021    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:46.976165 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:46.976135    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:47.093126 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:47.093092    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:47.950417 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:47.950377    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:48.044588 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:48.044553    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:48.147380 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:48.147350    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:49.550027 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:49.549995    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:55.448712 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:55.448678    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:55.514409 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:55.514353    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:21:56.157527 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:21:56.157492    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:02.328319 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:02.328284    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:02.438151 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:02.438121    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:02.524297 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:02.524264    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:13.065628 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:13.065598    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:13.433635 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:13.433603    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:15.347233 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:15.347201    2552 scope.go:117] "RemoveContainer" containerID="66c1dc37275a6f6da3d3dc3075ccdadb9c6bf32b9209860f8eab9707b1175178"
Jul 18 02:22:30.561847 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:30.561813    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:34.399694 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:34.399667    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:35.230088 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:35.230057    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:35.331405 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:35.331376    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:40.507049 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:40.507008    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:40.607232 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:40.607200    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:40.853618 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:40.853525    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:41.210364 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:41.210331    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:41.492355 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:41.492271    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:44.138090 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.138056    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:44.219141 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.219103    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"]
Jul 18 02:22:44.219778 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.219754    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f49cc4-d9db-4c6f-8f15-0ec1e824233c" containerName="revoke-keys"
Jul 18 02:22:44.219836 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.219784    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f49cc4-d9db-4c6f-8f15-0ec1e824233c" containerName="revoke-keys"
Jul 18 02:22:44.219836 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.219827    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3209c53d-801c-45d0-b917-142bd3f7533f" containerName="maas-api"
Jul 18 02:22:44.219907 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.219835    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3209c53d-801c-45d0-b917-142bd3f7533f" containerName="maas-api"
Jul 18 02:22:44.219950 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.219935    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f49cc4-d9db-4c6f-8f15-0ec1e824233c" containerName="revoke-keys"
Jul 18 02:22:44.219985 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.219956    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3209c53d-801c-45d0-b917-142bd3f7533f" containerName="maas-api"
Jul 18 02:22:44.223754 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.223734    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:44.225902 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.225880    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-0c89a1-serving-cert\""
Jul 18 02:22:44.230483 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.230459    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:44.235168 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.235121    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"]
Jul 18 02:22:44.293358 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.293316    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8595808-d0b2-4881-b6c5-58602625820b-maas-api-tls\") pod \"maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:44.293560 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.293400    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8595808-d0b2-4881-b6c5-58602625820b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:44.293560 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.293468    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7l7k\" (UniqueName: \"kubernetes.io/projected/c8595808-d0b2-4881-b6c5-58602625820b-kube-api-access-b7l7k\") pod \"maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:44.297715 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.297692    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:44.394282 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.394256    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8595808-d0b2-4881-b6c5-58602625820b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:44.394528 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.394309    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7l7k\" (UniqueName: \"kubernetes.io/projected/c8595808-d0b2-4881-b6c5-58602625820b-kube-api-access-b7l7k\") pod \"maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:44.394528 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.394346    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8595808-d0b2-4881-b6c5-58602625820b-maas-api-tls\") pod \"maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:44.394528 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:22:44.394475    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-serving-cert: secret "maas-api-e2e-inf-a-0c89a1-serving-cert" not found
Jul 18 02:22:44.394706 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:22:44.394544    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8595808-d0b2-4881-b6c5-58602625820b-maas-api-tls podName:c8595808-d0b2-4881-b6c5-58602625820b nodeName:}" failed. No retries permitted until 2026-07-18 02:22:44.894527919 +0000 UTC m=+2070.334024829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8595808-d0b2-4881-b6c5-58602625820b-maas-api-tls") pod "maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx" (UID: "c8595808-d0b2-4881-b6c5-58602625820b") : secret "maas-api-e2e-inf-a-0c89a1-serving-cert" not found
Jul 18 02:22:44.395018 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.394997    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8595808-d0b2-4881-b6c5-58602625820b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:44.403691 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.403667    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7l7k\" (UniqueName: \"kubernetes.io/projected/c8595808-d0b2-4881-b6c5-58602625820b-kube-api-access-b7l7k\") pod \"maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:44.630565 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.630535    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:44.804734 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.804651    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:44.899160 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.899130    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8595808-d0b2-4881-b6c5-58602625820b-maas-api-tls\") pod \"maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:44.901835 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.901810    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8595808-d0b2-4881-b6c5-58602625820b-maas-api-tls\") pod \"maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:44.905958 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:44.905933    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:45.136024 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:45.135933    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:45.283303 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:45.283275    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"]
Jul 18 02:22:45.285236 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:22:45.285207    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8595808_d0b2_4881_b6c5_58602625820b.slice/crio-f5f5515adf8f2a24ca327e4df46845770f48a1f8708a470767da8a5ecdba36dc WatchSource:0}: Error finding container f5f5515adf8f2a24ca327e4df46845770f48a1f8708a470767da8a5ecdba36dc: Status 404 returned error can't find the container with id f5f5515adf8f2a24ca327e4df46845770f48a1f8708a470767da8a5ecdba36dc
Jul 18 02:22:45.834949 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:45.834906    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx" event={"ID":"c8595808-d0b2-4881-b6c5-58602625820b","Type":"ContainerStarted","Data":"f5f5515adf8f2a24ca327e4df46845770f48a1f8708a470767da8a5ecdba36dc"}
Jul 18 02:22:46.842600 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:46.842566    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx" event={"ID":"c8595808-d0b2-4881-b6c5-58602625820b","Type":"ContainerStarted","Data":"04ea1033c076fc964b8a4a2aab70fddc22f98401a550af87c04c32d42db161a9"}
Jul 18 02:22:46.843002 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:46.842691    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:46.858755 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:46.858708    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx" podStartSLOduration=2.270605861 podStartE2EDuration="2.858694632s" totalImagesPullingTime="588.088771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:22:46.856822897 +0000 UTC m=+2072.296319828" watchObservedRunningTime="2026-07-18 02:22:46.858694632 +0000 UTC m=+2072.298191564"
Jul 18 02:22:48.667345 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:48.667317    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:49.015023 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:49.014930    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:49.488606 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:49.488575    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:49.596626 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:49.596595    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:52.263369 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.263331    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:52.351695 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.351649    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:52.532752 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.532666    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"]
Jul 18 02:22:52.536387 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.536369    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:52.540449 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.540409    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f0bd15-serving-cert\""
Jul 18 02:22:52.556505 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.556476    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"]
Jul 18 02:22:52.609010 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.608981    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:52.667558 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.667523    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65f61107-d96d-4ad7-92eb-d2ce46d38637-maas-api-tls\") pod \"maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:52.667736 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.667582    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f61107-d96d-4ad7-92eb-d2ce46d38637-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:52.667736 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.667632    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lchg\" (UniqueName: \"kubernetes.io/projected/65f61107-d96d-4ad7-92eb-d2ce46d38637-kube-api-access-8lchg\") pod \"maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:52.708050 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.708007    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:52.768940 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.768911    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65f61107-d96d-4ad7-92eb-d2ce46d38637-maas-api-tls\") pod \"maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:52.769108 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.768962    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f61107-d96d-4ad7-92eb-d2ce46d38637-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:52.769108 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.768981    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lchg\" (UniqueName: \"kubernetes.io/projected/65f61107-d96d-4ad7-92eb-d2ce46d38637-kube-api-access-8lchg\") pod \"maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:52.769668 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.769644    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f61107-d96d-4ad7-92eb-d2ce46d38637-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:52.771517 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.771493    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65f61107-d96d-4ad7-92eb-d2ce46d38637-maas-api-tls\") pod \"maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:52.784537 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.784470    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lchg\" (UniqueName: \"kubernetes.io/projected/65f61107-d96d-4ad7-92eb-d2ce46d38637-kube-api-access-8lchg\") pod \"maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:52.846321 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.846282    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:52.854372 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:52.854348    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:22:53.015985 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:53.015958    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"]
Jul 18 02:22:53.017825 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:22:53.017797    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f61107_d96d_4ad7_92eb_d2ce46d38637.slice/crio-1d8cd77846e71428e1927db610042c170a23e4ec3f2542f707fde7fbca367b7e WatchSource:0}: Error finding container 1d8cd77846e71428e1927db610042c170a23e4ec3f2542f707fde7fbca367b7e: Status 404 returned error can't find the container with id 1d8cd77846e71428e1927db610042c170a23e4ec3f2542f707fde7fbca367b7e
Jul 18 02:22:53.260278 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:53.260246    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:53.360472 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:53.360402    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:53.894445 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:53.894363    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6" event={"ID":"65f61107-d96d-4ad7-92eb-d2ce46d38637","Type":"ContainerStarted","Data":"7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519"}
Jul 18 02:22:53.894670 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:53.894465    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6" event={"ID":"65f61107-d96d-4ad7-92eb-d2ce46d38637","Type":"ContainerStarted","Data":"1d8cd77846e71428e1927db610042c170a23e4ec3f2542f707fde7fbca367b7e"}
Jul 18 02:22:53.894670 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:53.894611    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:22:53.915257 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:53.915205    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6" podStartSLOduration=1.493766479 podStartE2EDuration="1.915188367s" totalImagesPullingTime="421.421888ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:22:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:22:53.9135994 +0000 UTC m=+2079.353096344" watchObservedRunningTime="2026-07-18 02:22:53.915188367 +0000 UTC m=+2079.354685290"
Jul 18 02:22:56.682313 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:56.682279    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:22:59.905339 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:22:59.905310    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:23:05.834523 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:05.834490    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:05.934225 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:05.934194    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:06.392325 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:06.392294    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:14.110167 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:14.110136    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:14.206331 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:14.206293    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:15.199550 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:15.199520    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:23:15.207729 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:15.207707    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:23:15.369471 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:15.369414    2552 scope.go:117] "RemoveContainer" containerID="26071a0605328aee3eb29625bb2863e0631b9aba6147e4b01a0362b7939062a4"
Jul 18 02:23:15.378436 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:15.378397    2552 scope.go:117] "RemoveContainer" containerID="fd7ff9a3f3821b280ddde131b7dd556cbda2341ecb1e3340d6b983c07aab688a"
Jul 18 02:23:25.249097 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:25.249064    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:25.309962 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:25.309930    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:23:43.908999 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:23:43.908962    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:05.525675 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:05.525644    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:06.509788 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:06.509755    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:07.299769 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:07.299737    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:07.399296 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:07.399259    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:17.637679 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:17.637647    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:27.036881 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.036806    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"]
Jul 18 02:24:27.037370 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.037099    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx" podUID="c8595808-d0b2-4881-b6c5-58602625820b" containerName="maas-api" containerID="cri-o://04ea1033c076fc964b8a4a2aab70fddc22f98401a550af87c04c32d42db161a9" gracePeriod=30
Jul 18 02:24:27.069788 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.069750    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:27.439536 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.439497    2552 generic.go:356] "Generic (PLEG): container finished" podID="c8595808-d0b2-4881-b6c5-58602625820b" containerID="04ea1033c076fc964b8a4a2aab70fddc22f98401a550af87c04c32d42db161a9" exitCode=0
Jul 18 02:24:27.439716 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.439677    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx" event={"ID":"c8595808-d0b2-4881-b6c5-58602625820b","Type":"ContainerDied","Data":"04ea1033c076fc964b8a4a2aab70fddc22f98401a550af87c04c32d42db161a9"}
Jul 18 02:24:27.494596 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.494571    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:24:27.577359 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.577271    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8595808-d0b2-4881-b6c5-58602625820b-openshift-service-ca\") pod \"c8595808-d0b2-4881-b6c5-58602625820b\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") "
Jul 18 02:24:27.577359 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.577310    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8595808-d0b2-4881-b6c5-58602625820b-maas-api-tls\") pod \"c8595808-d0b2-4881-b6c5-58602625820b\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") "
Jul 18 02:24:27.577359 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.577348    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7l7k\" (UniqueName: \"kubernetes.io/projected/c8595808-d0b2-4881-b6c5-58602625820b-kube-api-access-b7l7k\") pod \"c8595808-d0b2-4881-b6c5-58602625820b\" (UID: \"c8595808-d0b2-4881-b6c5-58602625820b\") "
Jul 18 02:24:27.577786 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.577748    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8595808-d0b2-4881-b6c5-58602625820b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c8595808-d0b2-4881-b6c5-58602625820b" (UID: "c8595808-d0b2-4881-b6c5-58602625820b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:24:27.579647 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.579621    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8595808-d0b2-4881-b6c5-58602625820b-kube-api-access-b7l7k" (OuterVolumeSpecName: "kube-api-access-b7l7k") pod "c8595808-d0b2-4881-b6c5-58602625820b" (UID: "c8595808-d0b2-4881-b6c5-58602625820b"). InnerVolumeSpecName "kube-api-access-b7l7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:24:27.579734 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.579652    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8595808-d0b2-4881-b6c5-58602625820b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c8595808-d0b2-4881-b6c5-58602625820b" (UID: "c8595808-d0b2-4881-b6c5-58602625820b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:24:27.678407 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.678375    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8595808-d0b2-4881-b6c5-58602625820b-maas-api-tls\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:24:27.678407 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.678404    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7l7k\" (UniqueName: \"kubernetes.io/projected/c8595808-d0b2-4881-b6c5-58602625820b-kube-api-access-b7l7k\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:24:27.678407 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:27.678412    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8595808-d0b2-4881-b6c5-58602625820b-openshift-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:24:28.446768 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:28.446740    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"
Jul 18 02:24:28.447209 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:28.446776    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx" event={"ID":"c8595808-d0b2-4881-b6c5-58602625820b","Type":"ContainerDied","Data":"f5f5515adf8f2a24ca327e4df46845770f48a1f8708a470767da8a5ecdba36dc"}
Jul 18 02:24:28.447209 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:28.446846    2552 scope.go:117] "RemoveContainer" containerID="04ea1033c076fc964b8a4a2aab70fddc22f98401a550af87c04c32d42db161a9"
Jul 18 02:24:28.468527 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:28.468495    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"]
Jul 18 02:24:28.470894 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:28.470868    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c89a1-85656c9c59-qmzsx"]
Jul 18 02:24:29.076601 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:29.076570    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8595808-d0b2-4881-b6c5-58602625820b" path="/var/lib/kubelet/pods/c8595808-d0b2-4881-b6c5-58602625820b/volumes"
Jul 18 02:24:31.700213 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:31.700159    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:32.129549 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:32.129463    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:33.173642 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:33.173613    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:37.823280 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:37.823245    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:37.929781 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:37.929747    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:38.135163 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:38.135065    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:38.353244 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:38.353207    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:39.224982 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:39.224948    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:39.322588 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:39.322556    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:48.888278 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:48.888246    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"]
Jul 18 02:24:48.888820 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:48.888488    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6" podUID="65f61107-d96d-4ad7-92eb-d2ce46d38637" containerName="maas-api" containerID="cri-o://7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519" gracePeriod=30
Jul 18 02:24:48.918656 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:48.918625    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:49.346736 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.346708    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:24:49.435307 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.435219    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65f61107-d96d-4ad7-92eb-d2ce46d38637-maas-api-tls\") pod \"65f61107-d96d-4ad7-92eb-d2ce46d38637\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") "
Jul 18 02:24:49.435307 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.435265    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lchg\" (UniqueName: \"kubernetes.io/projected/65f61107-d96d-4ad7-92eb-d2ce46d38637-kube-api-access-8lchg\") pod \"65f61107-d96d-4ad7-92eb-d2ce46d38637\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") "
Jul 18 02:24:49.435307 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.435307    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f61107-d96d-4ad7-92eb-d2ce46d38637-openshift-service-ca\") pod \"65f61107-d96d-4ad7-92eb-d2ce46d38637\" (UID: \"65f61107-d96d-4ad7-92eb-d2ce46d38637\") "
Jul 18 02:24:49.435745 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.435720    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65f61107-d96d-4ad7-92eb-d2ce46d38637-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "65f61107-d96d-4ad7-92eb-d2ce46d38637" (UID: "65f61107-d96d-4ad7-92eb-d2ce46d38637"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:24:49.437579 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.437556    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65f61107-d96d-4ad7-92eb-d2ce46d38637-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "65f61107-d96d-4ad7-92eb-d2ce46d38637" (UID: "65f61107-d96d-4ad7-92eb-d2ce46d38637"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:24:49.437579 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.437569    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65f61107-d96d-4ad7-92eb-d2ce46d38637-kube-api-access-8lchg" (OuterVolumeSpecName: "kube-api-access-8lchg") pod "65f61107-d96d-4ad7-92eb-d2ce46d38637" (UID: "65f61107-d96d-4ad7-92eb-d2ce46d38637"). InnerVolumeSpecName "kube-api-access-8lchg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:24:49.536153 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.536116    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65f61107-d96d-4ad7-92eb-d2ce46d38637-maas-api-tls\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:24:49.536153 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.536150    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lchg\" (UniqueName: \"kubernetes.io/projected/65f61107-d96d-4ad7-92eb-d2ce46d38637-kube-api-access-8lchg\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:24:49.536360 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.536165    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f61107-d96d-4ad7-92eb-d2ce46d38637-openshift-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:24:49.580335 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.580304    2552 generic.go:356] "Generic (PLEG): container finished" podID="65f61107-d96d-4ad7-92eb-d2ce46d38637" containerID="7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519" exitCode=0
Jul 18 02:24:49.580527 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.580348    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"
Jul 18 02:24:49.580527 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.580387    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6" event={"ID":"65f61107-d96d-4ad7-92eb-d2ce46d38637","Type":"ContainerDied","Data":"7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519"}
Jul 18 02:24:49.580527 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.580464    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6" event={"ID":"65f61107-d96d-4ad7-92eb-d2ce46d38637","Type":"ContainerDied","Data":"1d8cd77846e71428e1927db610042c170a23e4ec3f2542f707fde7fbca367b7e"}
Jul 18 02:24:49.580527 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.580491    2552 scope.go:117] "RemoveContainer" containerID="7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519"
Jul 18 02:24:49.590347 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.590330    2552 scope.go:117] "RemoveContainer" containerID="7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519"
Jul 18 02:24:49.590629 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:24:49.590613    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519\": container with ID starting with 7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519 not found: ID does not exist" containerID="7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519"
Jul 18 02:24:49.590690 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.590637    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519"} err="failed to get container status \"7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519\": rpc error: code = NotFound desc = could not find container \"7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519\": container with ID starting with 7a9dae2f221b839332d8cc0275a61b8f85cf8ae88b945d77bd6cfd0e3236b519 not found: ID does not exist"
Jul 18 02:24:49.603624 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.603596    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"]
Jul 18 02:24:49.607608 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:49.607588    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f0bd15-c57645ff4-7dfj6"]
Jul 18 02:24:51.075849 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:51.075815    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65f61107-d96d-4ad7-92eb-d2ce46d38637" path="/var/lib/kubelet/pods/65f61107-d96d-4ad7-92eb-d2ce46d38637/volumes"
Jul 18 02:24:53.548546 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:53.548511    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:54.005415 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:54.005384    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:57.341855 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:57.341821    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:59.921619 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:59.921585    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:24:59.987437 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:24:59.987405    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:00.217471 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:00.217367    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:00.422536 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:00.422505    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:00.764249 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:00.764219    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:01.099550 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:01.099459    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:01.323632 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:01.323602    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:03.854333 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:03.854303    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:03.933300 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:03.933272    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:04.079608 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.079567    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"]
Jul 18 02:25:04.079996 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.079983    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8595808-d0b2-4881-b6c5-58602625820b" containerName="maas-api"
Jul 18 02:25:04.080051 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.080000    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8595808-d0b2-4881-b6c5-58602625820b" containerName="maas-api"
Jul 18 02:25:04.080051 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.080022    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65f61107-d96d-4ad7-92eb-d2ce46d38637" containerName="maas-api"
Jul 18 02:25:04.080051 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.080028    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="65f61107-d96d-4ad7-92eb-d2ce46d38637" containerName="maas-api"
Jul 18 02:25:04.080148 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.080104    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8595808-d0b2-4881-b6c5-58602625820b" containerName="maas-api"
Jul 18 02:25:04.080148 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.080114    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="65f61107-d96d-4ad7-92eb-d2ce46d38637" containerName="maas-api"
Jul 18 02:25:04.084642 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.084622    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:04.087149 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.087122    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-9a3654-serving-cert\""
Jul 18 02:25:04.102146 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.102107    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"]
Jul 18 02:25:04.126127 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.126040    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:04.154310 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.154278    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvrqz\" (UniqueName: \"kubernetes.io/projected/ac239020-2b5b-4230-a914-e8ba0013f8de-kube-api-access-bvrqz\") pod \"maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:04.154523 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.154327    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac239020-2b5b-4230-a914-e8ba0013f8de-openshift-service-ca\") pod \"maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:04.154523 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.154502    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac239020-2b5b-4230-a914-e8ba0013f8de-maas-api-tls\") pod \"maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:04.255446 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.255385    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac239020-2b5b-4230-a914-e8ba0013f8de-maas-api-tls\") pod \"maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:04.255633 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.255508    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvrqz\" (UniqueName: \"kubernetes.io/projected/ac239020-2b5b-4230-a914-e8ba0013f8de-kube-api-access-bvrqz\") pod \"maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:04.255633 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:25:04.255512    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-serving-cert: secret "maas-api-e2e-api-a-9a3654-serving-cert" not found
Jul 18 02:25:04.255633 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.255539    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac239020-2b5b-4230-a914-e8ba0013f8de-openshift-service-ca\") pod \"maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:04.255633 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:25:04.255574    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac239020-2b5b-4230-a914-e8ba0013f8de-maas-api-tls podName:ac239020-2b5b-4230-a914-e8ba0013f8de nodeName:}" failed. No retries permitted until 2026-07-18 02:25:04.755557714 +0000 UTC m=+2210.195054625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ac239020-2b5b-4230-a914-e8ba0013f8de-maas-api-tls") pod "maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf" (UID: "ac239020-2b5b-4230-a914-e8ba0013f8de") : secret "maas-api-e2e-api-a-9a3654-serving-cert" not found
Jul 18 02:25:04.256138 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.256121    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac239020-2b5b-4230-a914-e8ba0013f8de-openshift-service-ca\") pod \"maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:04.267598 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.267578    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvrqz\" (UniqueName: \"kubernetes.io/projected/ac239020-2b5b-4230-a914-e8ba0013f8de-kube-api-access-bvrqz\") pod \"maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:04.452465 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.452404    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:04.698774 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.698740    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:04.759875 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.759768    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac239020-2b5b-4230-a914-e8ba0013f8de-maas-api-tls\") pod \"maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:04.762527 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.762500    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac239020-2b5b-4230-a914-e8ba0013f8de-maas-api-tls\") pod \"maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:04.803248 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.803216    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:04.995166 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:04.995123    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:05.163665 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:05.163642    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"]
Jul 18 02:25:05.165722 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:25:05.165696    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac239020_2b5b_4230_a914_e8ba0013f8de.slice/crio-15e4d4f89eac353db70ba5ac2289eb7e374e7b990d074d84081dd994affc3dd3 WatchSource:0}: Error finding container 15e4d4f89eac353db70ba5ac2289eb7e374e7b990d074d84081dd994affc3dd3: Status 404 returned error can't find the container with id 15e4d4f89eac353db70ba5ac2289eb7e374e7b990d074d84081dd994affc3dd3
Jul 18 02:25:05.167022 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:05.167003    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:25:05.672995 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:05.672959    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf" event={"ID":"ac239020-2b5b-4230-a914-e8ba0013f8de","Type":"ContainerStarted","Data":"15e4d4f89eac353db70ba5ac2289eb7e374e7b990d074d84081dd994affc3dd3"}
Jul 18 02:25:06.679897 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:06.679862    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf" event={"ID":"ac239020-2b5b-4230-a914-e8ba0013f8de","Type":"ContainerStarted","Data":"467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad"}
Jul 18 02:25:06.680296 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:06.679925    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:06.697176 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:06.697123    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf" podStartSLOduration=2.129210436 podStartE2EDuration="2.697107779s" totalImagesPullingTime="567.897343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:25:06.694858724 +0000 UTC m=+2212.134355658" watchObservedRunningTime="2026-07-18 02:25:06.697107779 +0000 UTC m=+2212.136604712"
Jul 18 02:25:08.318096 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:08.318063    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:08.790392 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:08.790346    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:09.120286 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:09.120196    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:09.217644 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:09.217610    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:11.914163 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:11.914123    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:12.008387 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:12.008343    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:12.204371 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:12.204280    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:12.594702 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:12.594623    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:12.691171 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:12.691139    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:12.743300 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:12.743271    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:12.853313 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:12.853189    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:15.416001 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:15.415969    2552 scope.go:117] "RemoveContainer" containerID="402e19c5bd03b891b044c0cbd3d8c3e8ce12e96cf82bd404c367c1857b6c33a4"
Jul 18 02:25:16.413084 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:16.413027    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:25.090270 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:25.090237    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:25.190784 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:25.190746    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:33.730321 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:33.730286    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:33.832395 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:33.832331    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:34.955720 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:34.955667    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:35.056105 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:35.056062    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:40.176917 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.176821    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"]
Jul 18 02:25:40.179726 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.178057    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf" podUID="ac239020-2b5b-4230-a914-e8ba0013f8de" containerName="maas-api" containerID="cri-o://467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad" gracePeriod=30
Jul 18 02:25:40.199022 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.198999    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:40.428204 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.428139    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:40.584628 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.584587    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvrqz\" (UniqueName: \"kubernetes.io/projected/ac239020-2b5b-4230-a914-e8ba0013f8de-kube-api-access-bvrqz\") pod \"ac239020-2b5b-4230-a914-e8ba0013f8de\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") "
Jul 18 02:25:40.584845 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.584667    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac239020-2b5b-4230-a914-e8ba0013f8de-maas-api-tls\") pod \"ac239020-2b5b-4230-a914-e8ba0013f8de\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") "
Jul 18 02:25:40.584845 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.584730    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac239020-2b5b-4230-a914-e8ba0013f8de-openshift-service-ca\") pod \"ac239020-2b5b-4230-a914-e8ba0013f8de\" (UID: \"ac239020-2b5b-4230-a914-e8ba0013f8de\") "
Jul 18 02:25:40.585082 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.585056    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac239020-2b5b-4230-a914-e8ba0013f8de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ac239020-2b5b-4230-a914-e8ba0013f8de" (UID: "ac239020-2b5b-4230-a914-e8ba0013f8de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:25:40.586826 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.586806    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac239020-2b5b-4230-a914-e8ba0013f8de-kube-api-access-bvrqz" (OuterVolumeSpecName: "kube-api-access-bvrqz") pod "ac239020-2b5b-4230-a914-e8ba0013f8de" (UID: "ac239020-2b5b-4230-a914-e8ba0013f8de"). InnerVolumeSpecName "kube-api-access-bvrqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:25:40.586957 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.586937    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac239020-2b5b-4230-a914-e8ba0013f8de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ac239020-2b5b-4230-a914-e8ba0013f8de" (UID: "ac239020-2b5b-4230-a914-e8ba0013f8de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:25:40.685822 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.685731    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac239020-2b5b-4230-a914-e8ba0013f8de-maas-api-tls\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:25:40.685822 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.685762    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac239020-2b5b-4230-a914-e8ba0013f8de-openshift-service-ca\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:25:40.685822 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.685773    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvrqz\" (UniqueName: \"kubernetes.io/projected/ac239020-2b5b-4230-a914-e8ba0013f8de-kube-api-access-bvrqz\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:25:40.882318 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.882278    2552 generic.go:356] "Generic (PLEG): container finished" podID="ac239020-2b5b-4230-a914-e8ba0013f8de" containerID="467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad" exitCode=0
Jul 18 02:25:40.882513 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.882324    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"
Jul 18 02:25:40.882513 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.882337    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf" event={"ID":"ac239020-2b5b-4230-a914-e8ba0013f8de","Type":"ContainerDied","Data":"467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad"}
Jul 18 02:25:40.882513 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.882387    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf" event={"ID":"ac239020-2b5b-4230-a914-e8ba0013f8de","Type":"ContainerDied","Data":"15e4d4f89eac353db70ba5ac2289eb7e374e7b990d074d84081dd994affc3dd3"}
Jul 18 02:25:40.882513 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.882411    2552 scope.go:117] "RemoveContainer" containerID="467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad"
Jul 18 02:25:40.892307 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.892266    2552 scope.go:117] "RemoveContainer" containerID="467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad"
Jul 18 02:25:40.892558 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:25:40.892535    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad\": container with ID starting with 467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad not found: ID does not exist" containerID="467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad"
Jul 18 02:25:40.892625 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.892568    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad"} err="failed to get container status \"467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad\": rpc error: code = NotFound desc = could not find container \"467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad\": container with ID starting with 467f15c58ed6e0227d838689b320dbeae804c7a14d80192385e4f6fa688577ad not found: ID does not exist"
Jul 18 02:25:40.903892 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.903864    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"]
Jul 18 02:25:40.909352 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:40.909329    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9a3654-749ccff6c6-2ptnf"]
Jul 18 02:25:41.076657 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:41.076571    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac239020-2b5b-4230-a914-e8ba0013f8de" path="/var/lib/kubelet/pods/ac239020-2b5b-4230-a914-e8ba0013f8de/volumes"
Jul 18 02:25:44.555869 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:44.555831    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:44.949858 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:44.949832    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:50.604083 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:50.604045    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:50.703130 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:50.703097    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:50.874038 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:50.873954    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:51.160467 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:51.160409    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:51.971070 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:51.971034    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:25:52.072138 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:25:52.072112    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:06.686838 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:06.686798    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:11.288768 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:11.288731    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:11.699028 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:11.698991    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:17.770949 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:17.770907    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:17.829569 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:17.829533    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:18.077623 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:18.077537    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:18.323166 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:18.323134    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:18.727401 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:18.727368    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:19.036172 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:19.036086    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:19.332888 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:19.332802    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:22.326653 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:22.326620    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:22.455068 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:22.455035    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:22.599513 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:22.599405    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:23.206258 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:23.206219    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:23.294846 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:23.294809    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:23.400149 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:23.400112    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:26.839464 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:26.839411    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:27.186073 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:27.186037    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:27.488094 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:27.488016    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:27.767493 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:27.767362    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:30.300378 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:30.300343    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:30.411547 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:30.411513    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:30.531187 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:30.531129    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:31.085574 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:31.085543    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:31.185263 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:31.185218    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:34.817014 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:34.816983    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:43.951587 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:43.951551    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:44.056113 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:44.056076    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:44.407010 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:44.406965    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:52.754181 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:52.754140    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:26:52.852318 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:26:52.852281    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:03.760120 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.760086    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"]
Jul 18 02:27:03.760576 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.760568    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac239020-2b5b-4230-a914-e8ba0013f8de" containerName="maas-api"
Jul 18 02:27:03.760621 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.760581    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac239020-2b5b-4230-a914-e8ba0013f8de" containerName="maas-api"
Jul 18 02:27:03.760674 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.760663    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac239020-2b5b-4230-a914-e8ba0013f8de" containerName="maas-api"
Jul 18 02:27:03.763783 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.763767    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.766740 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.766694    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"openshift-service-ca.crt\""
Jul 18 02:27:03.766740 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.766703    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"default-dockercfg-9mnjr\""
Jul 18 02:27:03.766740 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.766717    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"kube-root-ca.crt\""
Jul 18 02:27:03.766976 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.766703    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"auth-test-model-2a8451-kserve-self-signed-certs\""
Jul 18 02:27:03.773302 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.773279    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"]
Jul 18 02:27:03.794063 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.794032    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:03.813002 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.812975    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckn9k\" (UniqueName: \"kubernetes.io/projected/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-kube-api-access-ckn9k\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.813162 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.813023    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tls-certs\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.813162 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.813055    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tmp-dir\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.813162 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.813120    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-model-cache\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.813162 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.813146    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-home\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.813321 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.813165    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-dshm\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.914332 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.914298    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-model-cache\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.914332 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.914335    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-home\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.914603 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.914352    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-dshm\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.914603 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.914410    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckn9k\" (UniqueName: \"kubernetes.io/projected/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-kube-api-access-ckn9k\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.914603 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.914468    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tls-certs\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.914603 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.914485    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tmp-dir\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.914826 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.914802    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-model-cache\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.914885 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.914833    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-home\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.915002 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.914977    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tmp-dir\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.916919 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.916892    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-dshm\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.917148 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.917129    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tls-certs\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:03.922285 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:03.922263    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckn9k\" (UniqueName: \"kubernetes.io/projected/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-kube-api-access-ckn9k\") pod \"auth-test-model-2a8451-kserve-759476495-tgmtz\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") " pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:04.074653 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:04.074572    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:04.217561 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:04.217530    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"]
Jul 18 02:27:04.219441 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:27:04.219393    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f0f0c23_dc57_453e_94b7_2a8296aefd5c.slice/crio-7aec83760d3d8558c960b135e49b8159f6b6ff621da415f99df7278a285101f7 WatchSource:0}: Error finding container 7aec83760d3d8558c960b135e49b8159f6b6ff621da415f99df7278a285101f7: Status 404 returned error can't find the container with id 7aec83760d3d8558c960b135e49b8159f6b6ff621da415f99df7278a285101f7
Jul 18 02:27:04.370164 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:04.370075    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz" event={"ID":"4f0f0c23-dc57-453e-94b7-2a8296aefd5c","Type":"ContainerStarted","Data":"d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be"}
Jul 18 02:27:04.370164 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:04.370119    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz" event={"ID":"4f0f0c23-dc57-453e-94b7-2a8296aefd5c","Type":"ContainerStarted","Data":"7aec83760d3d8558c960b135e49b8159f6b6ff621da415f99df7278a285101f7"}
Jul 18 02:27:04.388642 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:04.388591    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz" podStartSLOduration=1.388576739 podStartE2EDuration="1.388576739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:27:04.385907914 +0000 UTC m=+2329.825404846" watchObservedRunningTime="2026-07-18 02:27:04.388576739 +0000 UTC m=+2329.828073671"
Jul 18 02:27:14.074729 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:14.074685    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:14.074729 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:14.074739    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:14.087397 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:14.087369    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:14.442084 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:14.442053    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:27:15.473259 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:15.473217    2552 scope.go:117] "RemoveContainer" containerID="bb5b82d0f1f381d6502e90bb9fb3ac2f4f47a224be02cd8cf02e88d767293dd3"
Jul 18 02:27:33.674436 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:33.674390    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:34.662402 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:34.662369    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:36.119364 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:36.119334    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:27:36.220575 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:27:36.220541    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:28:15.237588 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:28:15.237545    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:28:15.247018 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:28:15.246992    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:29:02.745979 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:02.745898    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:29:03.863475 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:03.863414    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"]
Jul 18 02:29:03.870460 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:03.870415    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:29:12.480898 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:12.480856    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:29:18.884528 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:18.884483    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz" podUID="4f0f0c23-dc57-453e-94b7-2a8296aefd5c" containerName="main" containerID="cri-o://d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be" gracePeriod=45
Jul 18 02:29:19.144042 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.144019    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:29:19.147813 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.147789    2552 generic.go:356] "Generic (PLEG): container finished" podID="4f0f0c23-dc57-453e-94b7-2a8296aefd5c" containerID="d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be" exitCode=0
Jul 18 02:29:19.147944 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.147835    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"
Jul 18 02:29:19.147944 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.147871    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz" event={"ID":"4f0f0c23-dc57-453e-94b7-2a8296aefd5c","Type":"ContainerDied","Data":"d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be"}
Jul 18 02:29:19.147944 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.147916    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz" event={"ID":"4f0f0c23-dc57-453e-94b7-2a8296aefd5c","Type":"ContainerDied","Data":"7aec83760d3d8558c960b135e49b8159f6b6ff621da415f99df7278a285101f7"}
Jul 18 02:29:19.147944 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.147934    2552 scope.go:117] "RemoveContainer" containerID="d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be"
Jul 18 02:29:19.159973 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.159953    2552 scope.go:117] "RemoveContainer" containerID="d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be"
Jul 18 02:29:19.160304 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:29:19.160276    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be\": container with ID starting with d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be not found: ID does not exist" containerID="d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be"
Jul 18 02:29:19.160405 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.160317    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be"} err="failed to get container status \"d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be\": rpc error: code = NotFound desc = could not find container \"d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be\": container with ID starting with d7ffb4360ccae06b3b937c7f627825a758f657eb976547fe37d0a94bf63372be not found: ID does not exist"
Jul 18 02:29:19.258642 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.258613    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-model-cache\") pod \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") "
Jul 18 02:29:19.258642 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.258648    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-home\") pod \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") "
Jul 18 02:29:19.258904 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.258669    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tmp-dir\") pod \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") "
Jul 18 02:29:19.258904 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.258719    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-dshm\") pod \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") "
Jul 18 02:29:19.258904 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.258769    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tls-certs\") pod \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") "
Jul 18 02:29:19.258904 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.258837    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckn9k\" (UniqueName: \"kubernetes.io/projected/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-kube-api-access-ckn9k\") pod \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\" (UID: \"4f0f0c23-dc57-453e-94b7-2a8296aefd5c\") "
Jul 18 02:29:19.259109 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.258970    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f0f0c23-dc57-453e-94b7-2a8296aefd5c" (UID: "4f0f0c23-dc57-453e-94b7-2a8296aefd5c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:29:19.259109 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.258986    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-home" (OuterVolumeSpecName: "home") pod "4f0f0c23-dc57-453e-94b7-2a8296aefd5c" (UID: "4f0f0c23-dc57-453e-94b7-2a8296aefd5c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:29:19.259216 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.259111    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-model-cache\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:29:19.259216 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.259129    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-home\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:29:19.259313 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.259215    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f0f0c23-dc57-453e-94b7-2a8296aefd5c" (UID: "4f0f0c23-dc57-453e-94b7-2a8296aefd5c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:29:19.261262 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.261233    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-kube-api-access-ckn9k" (OuterVolumeSpecName: "kube-api-access-ckn9k") pod "4f0f0c23-dc57-453e-94b7-2a8296aefd5c" (UID: "4f0f0c23-dc57-453e-94b7-2a8296aefd5c"). InnerVolumeSpecName "kube-api-access-ckn9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:29:19.261382 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.261252    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-dshm" (OuterVolumeSpecName: "dshm") pod "4f0f0c23-dc57-453e-94b7-2a8296aefd5c" (UID: "4f0f0c23-dc57-453e-94b7-2a8296aefd5c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:29:19.261382 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.261297    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f0f0c23-dc57-453e-94b7-2a8296aefd5c" (UID: "4f0f0c23-dc57-453e-94b7-2a8296aefd5c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:29:19.360155 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.360114    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckn9k\" (UniqueName: \"kubernetes.io/projected/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-kube-api-access-ckn9k\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:29:19.360155 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.360145    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tmp-dir\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:29:19.360155 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.360155    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-dshm\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:29:19.360155 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.360165    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f0f0c23-dc57-453e-94b7-2a8296aefd5c-tls-certs\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:29:19.469705 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.469673    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"]
Jul 18 02:29:19.471894 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:19.471871    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/auth-test-model-2a8451-kserve-759476495-tgmtz"]
Jul 18 02:29:21.075997 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:21.075961    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f0f0c23-dc57-453e-94b7-2a8296aefd5c" path="/var/lib/kubelet/pods/4f0f0c23-dc57-453e-94b7-2a8296aefd5c/volumes"
Jul 18 02:29:31.667493 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.667450    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"]
Jul 18 02:29:31.667972 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.667872    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f0f0c23-dc57-453e-94b7-2a8296aefd5c" containerName="main"
Jul 18 02:29:31.667972 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.667884    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f0f0c23-dc57-453e-94b7-2a8296aefd5c" containerName="main"
Jul 18 02:29:31.667972 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.667947    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f0f0c23-dc57-453e-94b7-2a8296aefd5c" containerName="main"
Jul 18 02:29:31.671121 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.671100    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.673955 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.673925    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"openshift-service-ca.crt\""
Jul 18 02:29:31.673955 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.673941    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"default-dockercfg-9mnjr\""
Jul 18 02:29:31.674147 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.673925    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"kube-root-ca.crt\""
Jul 18 02:29:31.674147 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.673928    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"sub-test-model-2a8451-kserve-self-signed-certs\""
Jul 18 02:29:31.681191 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.681158    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"]
Jul 18 02:29:31.701178 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.701151    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:29:31.771169 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.771136    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-dshm\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.771369 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.771184    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv628\" (UniqueName: \"kubernetes.io/projected/2d5196ec-70e6-4bb0-947c-616d886cf4f1-kube-api-access-tv628\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.771369 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.771232    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-model-cache\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.771369 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.771269    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tmp-dir\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.771369 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.771299    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tls-certs\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.771587 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.771377    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-home\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.872123 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.872079    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-dshm\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.872344 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.872135    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv628\" (UniqueName: \"kubernetes.io/projected/2d5196ec-70e6-4bb0-947c-616d886cf4f1-kube-api-access-tv628\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.872344 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.872163    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-model-cache\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.872344 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.872188    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tmp-dir\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.872344 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.872211    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tls-certs\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.872344 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.872238    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-home\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.872685 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.872659    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tmp-dir\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.872685 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.872672    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-model-cache\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.872794 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.872730    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-home\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.874708 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.874678    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-dshm\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.874826 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.874799    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tls-certs\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.880590 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.880565    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv628\" (UniqueName: \"kubernetes.io/projected/2d5196ec-70e6-4bb0-947c-616d886cf4f1-kube-api-access-tv628\") pod \"sub-test-model-2a8451-kserve-577f58696f-r9n6p\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") " pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:31.984743 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:31.984648    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:32.123333 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:32.123301    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"]
Jul 18 02:29:32.124980 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:29:32.124950    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d5196ec_70e6_4bb0_947c_616d886cf4f1.slice/crio-d044583149d3191eaa067cc7a131a662a4ef4e84de2dd6fd3439f79220b58ff0 WatchSource:0}: Error finding container d044583149d3191eaa067cc7a131a662a4ef4e84de2dd6fd3439f79220b58ff0: Status 404 returned error can't find the container with id d044583149d3191eaa067cc7a131a662a4ef4e84de2dd6fd3439f79220b58ff0
Jul 18 02:29:32.233208 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:32.233153    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p" event={"ID":"2d5196ec-70e6-4bb0-947c-616d886cf4f1","Type":"ContainerStarted","Data":"8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16"}
Jul 18 02:29:32.233208 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:32.233205    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p" event={"ID":"2d5196ec-70e6-4bb0-947c-616d886cf4f1","Type":"ContainerStarted","Data":"d044583149d3191eaa067cc7a131a662a4ef4e84de2dd6fd3439f79220b58ff0"}
Jul 18 02:29:32.247565 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:32.247411    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p" podStartSLOduration=1.247394695 podStartE2EDuration="1.247394695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:29:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:29:32.246772423 +0000 UTC m=+2477.686269378" watchObservedRunningTime="2026-07-18 02:29:32.247394695 +0000 UTC m=+2477.686891626"
Jul 18 02:29:41.985173 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:41.985130    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:41.985769 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:41.985192    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:41.998062 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:41.998037    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:29:42.299263 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:29:42.299175    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:30:27.990448 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:27.990382    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:30:28.888777 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:28.888738    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"]
Jul 18 02:30:28.910681 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:28.910649    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:30:29.527716 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:29.527678    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:30:43.912204 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:43.912153    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p" podUID="2d5196ec-70e6-4bb0-947c-616d886cf4f1" containerName="main" containerID="cri-o://8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16" gracePeriod=45
Jul 18 02:30:43.965134 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:43.965105    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:30:44.176292 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.176228    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:30:44.344682 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.344647    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-model-cache\") pod \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") "
Jul 18 02:30:44.344883 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.344701    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tmp-dir\") pod \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") "
Jul 18 02:30:44.344883 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.344766    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tv628\" (UniqueName: \"kubernetes.io/projected/2d5196ec-70e6-4bb0-947c-616d886cf4f1-kube-api-access-tv628\") pod \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") "
Jul 18 02:30:44.344883 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.344803    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tls-certs\") pod \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") "
Jul 18 02:30:44.344883 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.344826    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-home\") pod \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") "
Jul 18 02:30:44.345110 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.344907    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-dshm\") pod \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\" (UID: \"2d5196ec-70e6-4bb0-947c-616d886cf4f1\") "
Jul 18 02:30:44.345110 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.344977    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "2d5196ec-70e6-4bb0-947c-616d886cf4f1" (UID: "2d5196ec-70e6-4bb0-947c-616d886cf4f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:44.345110 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.345063    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2d5196ec-70e6-4bb0-947c-616d886cf4f1" (UID: "2d5196ec-70e6-4bb0-947c-616d886cf4f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:44.345266 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.345232    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-home" (OuterVolumeSpecName: "home") pod "2d5196ec-70e6-4bb0-947c-616d886cf4f1" (UID: "2d5196ec-70e6-4bb0-947c-616d886cf4f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:44.345321 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.345289    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tmp-dir\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:30:44.345321 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.345311    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-model-cache\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:30:44.347171 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.347136    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2d5196ec-70e6-4bb0-947c-616d886cf4f1" (UID: "2d5196ec-70e6-4bb0-947c-616d886cf4f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:30:44.347282 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.347171    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-dshm" (OuterVolumeSpecName: "dshm") pod "2d5196ec-70e6-4bb0-947c-616d886cf4f1" (UID: "2d5196ec-70e6-4bb0-947c-616d886cf4f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:44.347282 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.347195    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d5196ec-70e6-4bb0-947c-616d886cf4f1-kube-api-access-tv628" (OuterVolumeSpecName: "kube-api-access-tv628") pod "2d5196ec-70e6-4bb0-947c-616d886cf4f1" (UID: "2d5196ec-70e6-4bb0-947c-616d886cf4f1"). InnerVolumeSpecName "kube-api-access-tv628". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:30:44.446302 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.446208    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv628\" (UniqueName: \"kubernetes.io/projected/2d5196ec-70e6-4bb0-947c-616d886cf4f1-kube-api-access-tv628\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:30:44.446302 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.446242    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5196ec-70e6-4bb0-947c-616d886cf4f1-tls-certs\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:30:44.446302 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.446253    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-home\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:30:44.446302 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.446261    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d5196ec-70e6-4bb0-947c-616d886cf4f1-dshm\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:30:44.642573 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.642538    2552 generic.go:356] "Generic (PLEG): container finished" podID="2d5196ec-70e6-4bb0-947c-616d886cf4f1" containerID="8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16" exitCode=0
Jul 18 02:30:44.642776 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.642590    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"
Jul 18 02:30:44.642776 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.642624    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p" event={"ID":"2d5196ec-70e6-4bb0-947c-616d886cf4f1","Type":"ContainerDied","Data":"8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16"}
Jul 18 02:30:44.642776 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.642669    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p" event={"ID":"2d5196ec-70e6-4bb0-947c-616d886cf4f1","Type":"ContainerDied","Data":"d044583149d3191eaa067cc7a131a662a4ef4e84de2dd6fd3439f79220b58ff0"}
Jul 18 02:30:44.642776 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.642683    2552 scope.go:117] "RemoveContainer" containerID="8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16"
Jul 18 02:30:44.652380 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.652360    2552 scope.go:117] "RemoveContainer" containerID="8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16"
Jul 18 02:30:44.652686 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:30:44.652664    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16\": container with ID starting with 8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16 not found: ID does not exist" containerID="8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16"
Jul 18 02:30:44.652786 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.652693    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16"} err="failed to get container status \"8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16\": rpc error: code = NotFound desc = could not find container \"8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16\": container with ID starting with 8e643b56ff05d201154959737c44c9cc8bb77c14d1d2ec9364cf5aa82f6cbf16 not found: ID does not exist"
Jul 18 02:30:44.673464 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.673416    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"]
Jul 18 02:30:44.678680 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:44.678660    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/sub-test-model-2a8451-kserve-577f58696f-r9n6p"]
Jul 18 02:30:45.078068 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:30:45.078038    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d5196ec-70e6-4bb0-947c-616d886cf4f1" path="/var/lib/kubelet/pods/2d5196ec-70e6-4bb0-947c-616d886cf4f1/volumes"
Jul 18 02:31:16.576476 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.576439    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"]
Jul 18 02:31:16.577042 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.577022    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d5196ec-70e6-4bb0-947c-616d886cf4f1" containerName="main"
Jul 18 02:31:16.577120 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.577045    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d5196ec-70e6-4bb0-947c-616d886cf4f1" containerName="main"
Jul 18 02:31:16.577194 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.577181    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d5196ec-70e6-4bb0-947c-616d886cf4f1" containerName="main"
Jul 18 02:31:16.581341 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.581320    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.584261 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.584234    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"default-dockercfg-9mnjr\""
Jul 18 02:31:16.584387 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.584238    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"rate-test-model-2a8451-kserve-self-signed-certs\""
Jul 18 02:31:16.584387 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.584242    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"openshift-service-ca.crt\""
Jul 18 02:31:16.584387 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.584275    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2a8451\"/\"kube-root-ca.crt\""
Jul 18 02:31:16.589269 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.588974    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"]
Jul 18 02:31:16.599218 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.599174    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:31:16.639598 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.639552    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-home\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.639598 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.639603    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1831732-c73e-426f-bb93-60dff070a65f-tls-certs\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.639828 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.639706    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-model-cache\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.639828 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.639751    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-tmp-dir\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.639828 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.639780    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-dshm\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.639947 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.639839    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcmjw\" (UniqueName: \"kubernetes.io/projected/f1831732-c73e-426f-bb93-60dff070a65f-kube-api-access-vcmjw\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.740591 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.740546    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcmjw\" (UniqueName: \"kubernetes.io/projected/f1831732-c73e-426f-bb93-60dff070a65f-kube-api-access-vcmjw\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.740799 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.740609    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-home\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.740799 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.740626    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1831732-c73e-426f-bb93-60dff070a65f-tls-certs\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.740799 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.740665    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-model-cache\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.740799 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.740691    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-tmp-dir\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.740799 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.740781    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-dshm\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.741108 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.741087    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-home\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.741160 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.741121    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-model-cache\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.741160 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.741140    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-tmp-dir\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.743036 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.743016    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-dshm\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.743331 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.743314    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1831732-c73e-426f-bb93-60dff070a65f-tls-certs\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.747643 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.747625    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcmjw\" (UniqueName: \"kubernetes.io/projected/f1831732-c73e-426f-bb93-60dff070a65f-kube-api-access-vcmjw\") pod \"rate-test-model-2a8451-kserve-568599559f-c4plz\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") " pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:16.892214 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:16.892184    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:17.040652 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:17.040624    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"]
Jul 18 02:31:17.042334 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:31:17.042309    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1831732_c73e_426f_bb93_60dff070a65f.slice/crio-132b50e90da927a1eef47601275e76c5abee618483feffdfc4542e50267022e6 WatchSource:0}: Error finding container 132b50e90da927a1eef47601275e76c5abee618483feffdfc4542e50267022e6: Status 404 returned error can't find the container with id 132b50e90da927a1eef47601275e76c5abee618483feffdfc4542e50267022e6
Jul 18 02:31:17.840467 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:17.840399    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz" event={"ID":"f1831732-c73e-426f-bb93-60dff070a65f","Type":"ContainerStarted","Data":"a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474"}
Jul 18 02:31:17.840467 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:17.840472    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz" event={"ID":"f1831732-c73e-426f-bb93-60dff070a65f","Type":"ContainerStarted","Data":"132b50e90da927a1eef47601275e76c5abee618483feffdfc4542e50267022e6"}
Jul 18 02:31:17.860365 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:17.860302    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz" podStartSLOduration=1.860282871 podStartE2EDuration="1.860282871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:31:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:31:17.859863419 +0000 UTC m=+2583.299360351" watchObservedRunningTime="2026-07-18 02:31:17.860282871 +0000 UTC m=+2583.299779811"
Jul 18 02:31:26.893037 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:26.893000    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:26.893527 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:26.893046    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:26.905728 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:26.905699    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:27.907504 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:27.907469    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:31:44.695983 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:44.695897    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:31:44.795653 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:44.795615    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:31:53.588086 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:53.588047    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:31:53.688889 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:53.688853    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:31:59.183472 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:31:59.183395    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:00.249998 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:00.249956    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"]
Jul 18 02:32:00.271704 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:00.271674    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:01.236455 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:01.236406    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7229-140527973706160"]
Jul 18 02:32:01.239968 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:01.239949    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973706160"
Jul 18 02:32:01.241928 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:01.241892    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-f4q4d\""
Jul 18 02:32:01.245776 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:01.245756    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973706160"]
Jul 18 02:32:01.352062 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:01.352018    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6rgg\" (UniqueName: \"kubernetes.io/projected/466af37d-b290-4337-91b2-2f9aac567047-kube-api-access-h6rgg\") pod \"test-curl-7229-140527973706160\" (UID: \"466af37d-b290-4337-91b2-2f9aac567047\") " pod="opendatahub/test-curl-7229-140527973706160"
Jul 18 02:32:01.452766 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:01.452728    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6rgg\" (UniqueName: \"kubernetes.io/projected/466af37d-b290-4337-91b2-2f9aac567047-kube-api-access-h6rgg\") pod \"test-curl-7229-140527973706160\" (UID: \"466af37d-b290-4337-91b2-2f9aac567047\") " pod="opendatahub/test-curl-7229-140527973706160"
Jul 18 02:32:01.460384 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:01.460354    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6rgg\" (UniqueName: \"kubernetes.io/projected/466af37d-b290-4337-91b2-2f9aac567047-kube-api-access-h6rgg\") pod \"test-curl-7229-140527973706160\" (UID: \"466af37d-b290-4337-91b2-2f9aac567047\") " pod="opendatahub/test-curl-7229-140527973706160"
Jul 18 02:32:01.552337 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:01.552246    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973706160"
Jul 18 02:32:01.690895 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:01.690871    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973706160"]
Jul 18 02:32:01.693639 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:32:01.693611    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod466af37d_b290_4337_91b2_2f9aac567047.slice/crio-e97a9e05b4772312a736121edacd4f70f523c79c7704588e525e9678f35ac337 WatchSource:0}: Error finding container e97a9e05b4772312a736121edacd4f70f523c79c7704588e525e9678f35ac337: Status 404 returned error can't find the container with id e97a9e05b4772312a736121edacd4f70f523c79c7704588e525e9678f35ac337
Jul 18 02:32:01.698945 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:01.698920    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:32:02.096217 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:02.096179    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973706160" event={"ID":"466af37d-b290-4337-91b2-2f9aac567047","Type":"ContainerStarted","Data":"e97a9e05b4772312a736121edacd4f70f523c79c7704588e525e9678f35ac337"}
Jul 18 02:32:03.108939 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:03.108893    2552 generic.go:356] "Generic (PLEG): container finished" podID="466af37d-b290-4337-91b2-2f9aac567047" containerID="fc63ac42a017287c4d23c30c28d9e550244e58a424f5ac9bd2bcae2c3d044717" exitCode=0
Jul 18 02:32:03.109469 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:03.108970    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973706160" event={"ID":"466af37d-b290-4337-91b2-2f9aac567047","Type":"ContainerDied","Data":"fc63ac42a017287c4d23c30c28d9e550244e58a424f5ac9bd2bcae2c3d044717"}
Jul 18 02:32:04.256780 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.256753    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973706160"
Jul 18 02:32:04.276707 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.276674    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7229-140527973706160_466af37d-b290-4337-91b2-2f9aac567047/test-curl-7229-140527973706160/0.log"
Jul 18 02:32:04.299990 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.299949    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7229-140527973706160"]
Jul 18 02:32:04.303680 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.303654    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7229-140527973706160"]
Jul 18 02:32:04.379188 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.379105    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6rgg\" (UniqueName: \"kubernetes.io/projected/466af37d-b290-4337-91b2-2f9aac567047-kube-api-access-h6rgg\") pod \"466af37d-b290-4337-91b2-2f9aac567047\" (UID: \"466af37d-b290-4337-91b2-2f9aac567047\") "
Jul 18 02:32:04.381265 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.381238    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/466af37d-b290-4337-91b2-2f9aac567047-kube-api-access-h6rgg" (OuterVolumeSpecName: "kube-api-access-h6rgg") pod "466af37d-b290-4337-91b2-2f9aac567047" (UID: "466af37d-b290-4337-91b2-2f9aac567047"). InnerVolumeSpecName "kube-api-access-h6rgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:32:04.430794 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.430754    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7229-140527973706288"]
Jul 18 02:32:04.431209 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.431196    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466af37d-b290-4337-91b2-2f9aac567047" containerName="test-curl-7229-140527973706160"
Jul 18 02:32:04.431258 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.431211    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="466af37d-b290-4337-91b2-2f9aac567047" containerName="test-curl-7229-140527973706160"
Jul 18 02:32:04.431293 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.431274    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="466af37d-b290-4337-91b2-2f9aac567047" containerName="test-curl-7229-140527973706160"
Jul 18 02:32:04.434484 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.434468    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973706288"
Jul 18 02:32:04.440085 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.440054    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973706288"]
Jul 18 02:32:04.479929 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.479883    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d94h4\" (UniqueName: \"kubernetes.io/projected/9637b423-1e6b-4d52-b0fb-53689fb93f63-kube-api-access-d94h4\") pod \"test-curl-7229-140527973706288\" (UID: \"9637b423-1e6b-4d52-b0fb-53689fb93f63\") " pod="opendatahub/test-curl-7229-140527973706288"
Jul 18 02:32:04.480134 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.480074    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6rgg\" (UniqueName: \"kubernetes.io/projected/466af37d-b290-4337-91b2-2f9aac567047-kube-api-access-h6rgg\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:04.580545 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.580501    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d94h4\" (UniqueName: \"kubernetes.io/projected/9637b423-1e6b-4d52-b0fb-53689fb93f63-kube-api-access-d94h4\") pod \"test-curl-7229-140527973706288\" (UID: \"9637b423-1e6b-4d52-b0fb-53689fb93f63\") " pod="opendatahub/test-curl-7229-140527973706288"
Jul 18 02:32:04.588570 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.588535    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d94h4\" (UniqueName: \"kubernetes.io/projected/9637b423-1e6b-4d52-b0fb-53689fb93f63-kube-api-access-d94h4\") pod \"test-curl-7229-140527973706288\" (UID: \"9637b423-1e6b-4d52-b0fb-53689fb93f63\") " pod="opendatahub/test-curl-7229-140527973706288"
Jul 18 02:32:04.745375 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.745333    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973706288"
Jul 18 02:32:04.884313 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:04.884289    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973706288"]
Jul 18 02:32:04.885263 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:32:04.885238    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9637b423_1e6b_4d52_b0fb_53689fb93f63.slice/crio-3b55ba55d35ba4bbf0bf46e673126ae281f82090f70b5bb6d2f228a44fd7ce0c WatchSource:0}: Error finding container 3b55ba55d35ba4bbf0bf46e673126ae281f82090f70b5bb6d2f228a44fd7ce0c: Status 404 returned error can't find the container with id 3b55ba55d35ba4bbf0bf46e673126ae281f82090f70b5bb6d2f228a44fd7ce0c
Jul 18 02:32:05.077132 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:05.077053    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="466af37d-b290-4337-91b2-2f9aac567047" path="/var/lib/kubelet/pods/466af37d-b290-4337-91b2-2f9aac567047/volumes"
Jul 18 02:32:05.121834 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:05.121791    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973706288" event={"ID":"9637b423-1e6b-4d52-b0fb-53689fb93f63","Type":"ContainerStarted","Data":"3b55ba55d35ba4bbf0bf46e673126ae281f82090f70b5bb6d2f228a44fd7ce0c"}
Jul 18 02:32:05.123084 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:05.123063    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973706160"
Jul 18 02:32:05.123202 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:05.123068    2552 scope.go:117] "RemoveContainer" containerID="fc63ac42a017287c4d23c30c28d9e550244e58a424f5ac9bd2bcae2c3d044717"
Jul 18 02:32:06.132181 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:06.132141    2552 generic.go:356] "Generic (PLEG): container finished" podID="9637b423-1e6b-4d52-b0fb-53689fb93f63" containerID="947f0ff13fc646fd3f7c89f191a6d48725475205b9d5b443113690ffb8d0ffe8" exitCode=0
Jul 18 02:32:06.132616 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:06.132222    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973706288" event={"ID":"9637b423-1e6b-4d52-b0fb-53689fb93f63","Type":"ContainerDied","Data":"947f0ff13fc646fd3f7c89f191a6d48725475205b9d5b443113690ffb8d0ffe8"}
Jul 18 02:32:07.277186 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.277161    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973706288"
Jul 18 02:32:07.295538 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.295510    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7229-140527973706288_9637b423-1e6b-4d52-b0fb-53689fb93f63/test-curl-7229-140527973706288/0.log"
Jul 18 02:32:07.319891 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.319860    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7229-140527973706288"]
Jul 18 02:32:07.323638 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.323613    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7229-140527973706288"]
Jul 18 02:32:07.408399 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.408367    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d94h4\" (UniqueName: \"kubernetes.io/projected/9637b423-1e6b-4d52-b0fb-53689fb93f63-kube-api-access-d94h4\") pod \"9637b423-1e6b-4d52-b0fb-53689fb93f63\" (UID: \"9637b423-1e6b-4d52-b0fb-53689fb93f63\") "
Jul 18 02:32:07.418731 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.418639    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9637b423-1e6b-4d52-b0fb-53689fb93f63-kube-api-access-d94h4" (OuterVolumeSpecName: "kube-api-access-d94h4") pod "9637b423-1e6b-4d52-b0fb-53689fb93f63" (UID: "9637b423-1e6b-4d52-b0fb-53689fb93f63"). InnerVolumeSpecName "kube-api-access-d94h4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:32:07.485589 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.485547    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7229-140527973827472"]
Jul 18 02:32:07.486045 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.486027    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9637b423-1e6b-4d52-b0fb-53689fb93f63" containerName="test-curl-7229-140527973706288"
Jul 18 02:32:07.486126 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.486049    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9637b423-1e6b-4d52-b0fb-53689fb93f63" containerName="test-curl-7229-140527973706288"
Jul 18 02:32:07.486203 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.486186    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="9637b423-1e6b-4d52-b0fb-53689fb93f63" containerName="test-curl-7229-140527973706288"
Jul 18 02:32:07.489232 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.489211    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:07.494920 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.494896    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973827472"]
Jul 18 02:32:07.509354 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.509323    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d94h4\" (UniqueName: \"kubernetes.io/projected/9637b423-1e6b-4d52-b0fb-53689fb93f63-kube-api-access-d94h4\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:07.609907 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.609877    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjw2r\" (UniqueName: \"kubernetes.io/projected/1c06fe70-4698-4db1-a608-8ab4a520f905-kube-api-access-zjw2r\") pod \"test-curl-7229-140527973827472\" (UID: \"1c06fe70-4698-4db1-a608-8ab4a520f905\") " pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:07.711382 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.711285    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjw2r\" (UniqueName: \"kubernetes.io/projected/1c06fe70-4698-4db1-a608-8ab4a520f905-kube-api-access-zjw2r\") pod \"test-curl-7229-140527973827472\" (UID: \"1c06fe70-4698-4db1-a608-8ab4a520f905\") " pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:07.719754 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.719724    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjw2r\" (UniqueName: \"kubernetes.io/projected/1c06fe70-4698-4db1-a608-8ab4a520f905-kube-api-access-zjw2r\") pod \"test-curl-7229-140527973827472\" (UID: \"1c06fe70-4698-4db1-a608-8ab4a520f905\") " pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:07.800752 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.800719    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:07.945292 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:07.945264    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973827472"]
Jul 18 02:32:07.946236 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:32:07.946214    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c06fe70_4698_4db1_a608_8ab4a520f905.slice/crio-64f1390d6d2a729f8479f9cd5b5e071fc3353a2bfa74cef8d46cce6efbeadc63 WatchSource:0}: Error finding container 64f1390d6d2a729f8479f9cd5b5e071fc3353a2bfa74cef8d46cce6efbeadc63: Status 404 returned error can't find the container with id 64f1390d6d2a729f8479f9cd5b5e071fc3353a2bfa74cef8d46cce6efbeadc63
Jul 18 02:32:08.145698 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:08.145660    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973827472" event={"ID":"1c06fe70-4698-4db1-a608-8ab4a520f905","Type":"ContainerStarted","Data":"64f1390d6d2a729f8479f9cd5b5e071fc3353a2bfa74cef8d46cce6efbeadc63"}
Jul 18 02:32:08.146977 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:08.146960    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b55ba55d35ba4bbf0bf46e673126ae281f82090f70b5bb6d2f228a44fd7ce0c"
Jul 18 02:32:08.147083 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:08.146992    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973706288"
Jul 18 02:32:08.157166 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:32:08.157139    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7229-140527973706288\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="9637b423-1e6b-4d52-b0fb-53689fb93f63" pod="opendatahub/test-curl-7229-140527973706288"
Jul 18 02:32:09.077186 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:09.077153    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9637b423-1e6b-4d52-b0fb-53689fb93f63" path="/var/lib/kubelet/pods/9637b423-1e6b-4d52-b0fb-53689fb93f63/volumes"
Jul 18 02:32:09.154794 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:09.154756    2552 generic.go:356] "Generic (PLEG): container finished" podID="1c06fe70-4698-4db1-a608-8ab4a520f905" containerID="92085bdfe9c61041c9281895cdc66f822f8be25ff3550f4d1850a79b9e274a03" exitCode=0
Jul 18 02:32:09.154967 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:09.154820    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973827472" event={"ID":"1c06fe70-4698-4db1-a608-8ab4a520f905","Type":"ContainerDied","Data":"92085bdfe9c61041c9281895cdc66f822f8be25ff3550f4d1850a79b9e274a03"}
Jul 18 02:32:10.303886 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.303861    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:10.322618 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.322591    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7229-140527973827472_1c06fe70-4698-4db1-a608-8ab4a520f905/test-curl-7229-140527973827472/0.log"
Jul 18 02:32:10.349544 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.349507    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7229-140527973827472"]
Jul 18 02:32:10.352845 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.352813    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7229-140527973827472"]
Jul 18 02:32:10.435881 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.435801    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjw2r\" (UniqueName: \"kubernetes.io/projected/1c06fe70-4698-4db1-a608-8ab4a520f905-kube-api-access-zjw2r\") pod \"1c06fe70-4698-4db1-a608-8ab4a520f905\" (UID: \"1c06fe70-4698-4db1-a608-8ab4a520f905\") "
Jul 18 02:32:10.438048 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.438015    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c06fe70-4698-4db1-a608-8ab4a520f905-kube-api-access-zjw2r" (OuterVolumeSpecName: "kube-api-access-zjw2r") pod "1c06fe70-4698-4db1-a608-8ab4a520f905" (UID: "1c06fe70-4698-4db1-a608-8ab4a520f905"). InnerVolumeSpecName "kube-api-access-zjw2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:32:10.470018 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.469991    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7229-140527973827472"]
Jul 18 02:32:10.470466 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.470452    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c06fe70-4698-4db1-a608-8ab4a520f905" containerName="test-curl-7229-140527973827472"
Jul 18 02:32:10.470512 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.470468    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c06fe70-4698-4db1-a608-8ab4a520f905" containerName="test-curl-7229-140527973827472"
Jul 18 02:32:10.470597 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.470587    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c06fe70-4698-4db1-a608-8ab4a520f905" containerName="test-curl-7229-140527973827472"
Jul 18 02:32:10.473618 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.473604    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:10.479368 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.479346    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973827472"]
Jul 18 02:32:10.537177 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.537145    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk85v\" (UniqueName: \"kubernetes.io/projected/f11c7a51-9768-442e-878d-939d81252e56-kube-api-access-xk85v\") pod \"test-curl-7229-140527973827472\" (UID: \"f11c7a51-9768-442e-878d-939d81252e56\") " pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:10.537363 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.537271    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjw2r\" (UniqueName: \"kubernetes.io/projected/1c06fe70-4698-4db1-a608-8ab4a520f905-kube-api-access-zjw2r\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:10.637988 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.637943    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk85v\" (UniqueName: \"kubernetes.io/projected/f11c7a51-9768-442e-878d-939d81252e56-kube-api-access-xk85v\") pod \"test-curl-7229-140527973827472\" (UID: \"f11c7a51-9768-442e-878d-939d81252e56\") " pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:10.646389 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.646355    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk85v\" (UniqueName: \"kubernetes.io/projected/f11c7a51-9768-442e-878d-939d81252e56-kube-api-access-xk85v\") pod \"test-curl-7229-140527973827472\" (UID: \"f11c7a51-9768-442e-878d-939d81252e56\") " pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:10.784162 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.784069    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:10.923962 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:10.923933    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973827472"]
Jul 18 02:32:10.925242 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:32:10.925202    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf11c7a51_9768_442e_878d_939d81252e56.slice/crio-a27c831ffc321c3747f35448f88c20259f8a83e5fc1afca7a233822ec8e9b12f WatchSource:0}: Error finding container a27c831ffc321c3747f35448f88c20259f8a83e5fc1afca7a233822ec8e9b12f: Status 404 returned error can't find the container with id a27c831ffc321c3747f35448f88c20259f8a83e5fc1afca7a233822ec8e9b12f
Jul 18 02:32:11.076194 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:11.076116    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c06fe70-4698-4db1-a608-8ab4a520f905" path="/var/lib/kubelet/pods/1c06fe70-4698-4db1-a608-8ab4a520f905/volumes"
Jul 18 02:32:11.166907 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:11.166870    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973827472" event={"ID":"f11c7a51-9768-442e-878d-939d81252e56","Type":"ContainerStarted","Data":"a27c831ffc321c3747f35448f88c20259f8a83e5fc1afca7a233822ec8e9b12f"}
Jul 18 02:32:11.168132 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:11.168112    2552 scope.go:117] "RemoveContainer" containerID="92085bdfe9c61041c9281895cdc66f822f8be25ff3550f4d1850a79b9e274a03"
Jul 18 02:32:11.168246 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:11.168145    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:12.175978 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:12.175949    2552 generic.go:356] "Generic (PLEG): container finished" podID="f11c7a51-9768-442e-878d-939d81252e56" containerID="a7fbd21313a6003dbf4e353fda44165a63a09e4f18728105751f6ef3c3b85c8e" exitCode=0
Jul 18 02:32:12.176442 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:12.176032    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973827472" event={"ID":"f11c7a51-9768-442e-878d-939d81252e56","Type":"ContainerDied","Data":"a7fbd21313a6003dbf4e353fda44165a63a09e4f18728105751f6ef3c3b85c8e"}
Jul 18 02:32:13.324958 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.324933    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:13.346705 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.346653    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7229-140527973827472_f11c7a51-9768-442e-878d-939d81252e56/test-curl-7229-140527973827472/0.log"
Jul 18 02:32:13.373299 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.373271    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7229-140527973827472"]
Jul 18 02:32:13.377043 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.377015    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7229-140527973827472"]
Jul 18 02:32:13.466027 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.465919    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk85v\" (UniqueName: \"kubernetes.io/projected/f11c7a51-9768-442e-878d-939d81252e56-kube-api-access-xk85v\") pod \"f11c7a51-9768-442e-878d-939d81252e56\" (UID: \"f11c7a51-9768-442e-878d-939d81252e56\") "
Jul 18 02:32:13.468146 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.468109    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f11c7a51-9768-442e-878d-939d81252e56-kube-api-access-xk85v" (OuterVolumeSpecName: "kube-api-access-xk85v") pod "f11c7a51-9768-442e-878d-939d81252e56" (UID: "f11c7a51-9768-442e-878d-939d81252e56"). InnerVolumeSpecName "kube-api-access-xk85v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:32:13.504652 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.504621    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7229-140527973828480"]
Jul 18 02:32:13.505048 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.505035    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f11c7a51-9768-442e-878d-939d81252e56" containerName="test-curl-7229-140527973827472"
Jul 18 02:32:13.505088 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.505050    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f11c7a51-9768-442e-878d-939d81252e56" containerName="test-curl-7229-140527973827472"
Jul 18 02:32:13.505125 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.505116    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f11c7a51-9768-442e-878d-939d81252e56" containerName="test-curl-7229-140527973827472"
Jul 18 02:32:13.508084 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.508059    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:13.512695 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.512670    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973828480"]
Jul 18 02:32:13.566835 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.566803    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk85v\" (UniqueName: \"kubernetes.io/projected/f11c7a51-9768-442e-878d-939d81252e56-kube-api-access-xk85v\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:13.667912 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.667876    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g62qq\" (UniqueName: \"kubernetes.io/projected/cae76225-9527-4aed-a527-76d938b576da-kube-api-access-g62qq\") pod \"test-curl-7229-140527973828480\" (UID: \"cae76225-9527-4aed-a527-76d938b576da\") " pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:13.768659 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.768573    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g62qq\" (UniqueName: \"kubernetes.io/projected/cae76225-9527-4aed-a527-76d938b576da-kube-api-access-g62qq\") pod \"test-curl-7229-140527973828480\" (UID: \"cae76225-9527-4aed-a527-76d938b576da\") " pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:13.776404 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.776373    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g62qq\" (UniqueName: \"kubernetes.io/projected/cae76225-9527-4aed-a527-76d938b576da-kube-api-access-g62qq\") pod \"test-curl-7229-140527973828480\" (UID: \"cae76225-9527-4aed-a527-76d938b576da\") " pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:13.819985 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.819948    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:13.960481 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:13.960457    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973828480"]
Jul 18 02:32:13.961704 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:32:13.961680    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcae76225_9527_4aed_a527_76d938b576da.slice/crio-bbd0baad5a88d2edcd44bf7e1f7eed4b6e68b45e8e8616f09566dc7fd212de40 WatchSource:0}: Error finding container bbd0baad5a88d2edcd44bf7e1f7eed4b6e68b45e8e8616f09566dc7fd212de40: Status 404 returned error can't find the container with id bbd0baad5a88d2edcd44bf7e1f7eed4b6e68b45e8e8616f09566dc7fd212de40
Jul 18 02:32:14.189896 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:14.189868    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:14.190089 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:14.189909    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a27c831ffc321c3747f35448f88c20259f8a83e5fc1afca7a233822ec8e9b12f"
Jul 18 02:32:14.191151 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:14.191124    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973828480" event={"ID":"cae76225-9527-4aed-a527-76d938b576da","Type":"ContainerStarted","Data":"bbd0baad5a88d2edcd44bf7e1f7eed4b6e68b45e8e8616f09566dc7fd212de40"}
Jul 18 02:32:14.200125 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:32:14.200095    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7229-140527973827472\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="f11c7a51-9768-442e-878d-939d81252e56" pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:15.076344 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:32:15.076303    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7229-140527973827472\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="f11c7a51-9768-442e-878d-939d81252e56" pod="opendatahub/test-curl-7229-140527973827472"
Jul 18 02:32:15.078028 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.078006    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f11c7a51-9768-442e-878d-939d81252e56" path="/var/lib/kubelet/pods/f11c7a51-9768-442e-878d-939d81252e56/volumes"
Jul 18 02:32:15.198531 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.198499    2552 generic.go:356] "Generic (PLEG): container finished" podID="cae76225-9527-4aed-a527-76d938b576da" containerID="4ae25e648393d86c62290722e9f29f020948ca760e11afc5bd3d74de13a5bc98" exitCode=0
Jul 18 02:32:15.198717 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.198594    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973828480" event={"ID":"cae76225-9527-4aed-a527-76d938b576da","Type":"ContainerDied","Data":"4ae25e648393d86c62290722e9f29f020948ca760e11afc5bd3d74de13a5bc98"}
Jul 18 02:32:15.273372 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.273330    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz" podUID="f1831732-c73e-426f-bb93-60dff070a65f" containerName="main" containerID="cri-o://a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474" gracePeriod=45
Jul 18 02:32:15.542664 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.542635    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:32:15.686020 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.685931    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcmjw\" (UniqueName: \"kubernetes.io/projected/f1831732-c73e-426f-bb93-60dff070a65f-kube-api-access-vcmjw\") pod \"f1831732-c73e-426f-bb93-60dff070a65f\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") "
Jul 18 02:32:15.686020 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.685976    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-dshm\") pod \"f1831732-c73e-426f-bb93-60dff070a65f\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") "
Jul 18 02:32:15.686020 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.685999    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-model-cache\") pod \"f1831732-c73e-426f-bb93-60dff070a65f\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") "
Jul 18 02:32:15.686302 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.686039    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-home\") pod \"f1831732-c73e-426f-bb93-60dff070a65f\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") "
Jul 18 02:32:15.686302 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.686058    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1831732-c73e-426f-bb93-60dff070a65f-tls-certs\") pod \"f1831732-c73e-426f-bb93-60dff070a65f\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") "
Jul 18 02:32:15.686302 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.686073    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-tmp-dir\") pod \"f1831732-c73e-426f-bb93-60dff070a65f\" (UID: \"f1831732-c73e-426f-bb93-60dff070a65f\") "
Jul 18 02:32:15.686503 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.686327    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-home" (OuterVolumeSpecName: "home") pod "f1831732-c73e-426f-bb93-60dff070a65f" (UID: "f1831732-c73e-426f-bb93-60dff070a65f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:32:15.686503 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.686342    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-model-cache" (OuterVolumeSpecName: "model-cache") pod "f1831732-c73e-426f-bb93-60dff070a65f" (UID: "f1831732-c73e-426f-bb93-60dff070a65f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:32:15.686503 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.686418    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f1831732-c73e-426f-bb93-60dff070a65f" (UID: "f1831732-c73e-426f-bb93-60dff070a65f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:32:15.688483 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.688452    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1831732-c73e-426f-bb93-60dff070a65f-kube-api-access-vcmjw" (OuterVolumeSpecName: "kube-api-access-vcmjw") pod "f1831732-c73e-426f-bb93-60dff070a65f" (UID: "f1831732-c73e-426f-bb93-60dff070a65f"). InnerVolumeSpecName "kube-api-access-vcmjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:32:15.688483 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.688459    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1831732-c73e-426f-bb93-60dff070a65f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f1831732-c73e-426f-bb93-60dff070a65f" (UID: "f1831732-c73e-426f-bb93-60dff070a65f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:32:15.688610 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.688476    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-dshm" (OuterVolumeSpecName: "dshm") pod "f1831732-c73e-426f-bb93-60dff070a65f" (UID: "f1831732-c73e-426f-bb93-60dff070a65f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:32:15.787363 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.787324    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcmjw\" (UniqueName: \"kubernetes.io/projected/f1831732-c73e-426f-bb93-60dff070a65f-kube-api-access-vcmjw\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:15.787363 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.787357    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-dshm\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:15.787363 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.787370    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-model-cache\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:15.787704 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.787381    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-home\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:15.787704 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.787393    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1831732-c73e-426f-bb93-60dff070a65f-tls-certs\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:15.787704 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:15.787402    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1831732-c73e-426f-bb93-60dff070a65f-tmp-dir\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:16.206215 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.206178    2552 generic.go:356] "Generic (PLEG): container finished" podID="f1831732-c73e-426f-bb93-60dff070a65f" containerID="a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474" exitCode=0
Jul 18 02:32:16.206699 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.206252    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"
Jul 18 02:32:16.206699 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.206265    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz" event={"ID":"f1831732-c73e-426f-bb93-60dff070a65f","Type":"ContainerDied","Data":"a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474"}
Jul 18 02:32:16.206699 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.206554    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz" event={"ID":"f1831732-c73e-426f-bb93-60dff070a65f","Type":"ContainerDied","Data":"132b50e90da927a1eef47601275e76c5abee618483feffdfc4542e50267022e6"}
Jul 18 02:32:16.206699 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.206577    2552 scope.go:117] "RemoveContainer" containerID="a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474"
Jul 18 02:32:16.219483 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.219461    2552 scope.go:117] "RemoveContainer" containerID="a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474"
Jul 18 02:32:16.219806 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:32:16.219769    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474\": container with ID starting with a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474 not found: ID does not exist" containerID="a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474"
Jul 18 02:32:16.219911 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.219808    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474"} err="failed to get container status \"a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474\": rpc error: code = NotFound desc = could not find container \"a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474\": container with ID starting with a62f72bf5b6c22141dd629c612663ca2842a75b0a5ca33a16575efa7d0ffe474 not found: ID does not exist"
Jul 18 02:32:16.228748 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.228722    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"]
Jul 18 02:32:16.232172 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.232148    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2a8451/rate-test-model-2a8451-kserve-568599559f-c4plz"]
Jul 18 02:32:16.344283 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.344258    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:16.364830 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.364802    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7229-140527973828480_cae76225-9527-4aed-a527-76d938b576da/test-curl-7229-140527973828480/0.log"
Jul 18 02:32:16.391526 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.391494    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7229-140527973828480"]
Jul 18 02:32:16.395361 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.395338    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7229-140527973828480"]
Jul 18 02:32:16.494638 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.494544    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g62qq\" (UniqueName: \"kubernetes.io/projected/cae76225-9527-4aed-a527-76d938b576da-kube-api-access-g62qq\") pod \"cae76225-9527-4aed-a527-76d938b576da\" (UID: \"cae76225-9527-4aed-a527-76d938b576da\") "
Jul 18 02:32:16.496716 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.496687    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cae76225-9527-4aed-a527-76d938b576da-kube-api-access-g62qq" (OuterVolumeSpecName: "kube-api-access-g62qq") pod "cae76225-9527-4aed-a527-76d938b576da" (UID: "cae76225-9527-4aed-a527-76d938b576da"). InnerVolumeSpecName "kube-api-access-g62qq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:32:16.520988 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.520956    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7229-140527973828480"]
Jul 18 02:32:16.521448 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.521414    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cae76225-9527-4aed-a527-76d938b576da" containerName="test-curl-7229-140527973828480"
Jul 18 02:32:16.521544 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.521450    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="cae76225-9527-4aed-a527-76d938b576da" containerName="test-curl-7229-140527973828480"
Jul 18 02:32:16.521544 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.521465    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1831732-c73e-426f-bb93-60dff070a65f" containerName="main"
Jul 18 02:32:16.521544 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.521474    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1831732-c73e-426f-bb93-60dff070a65f" containerName="main"
Jul 18 02:32:16.521695 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.521581    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1831732-c73e-426f-bb93-60dff070a65f" containerName="main"
Jul 18 02:32:16.521695 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.521596    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="cae76225-9527-4aed-a527-76d938b576da" containerName="test-curl-7229-140527973828480"
Jul 18 02:32:16.524840 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.524821    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:16.529955 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.529927    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973828480"]
Jul 18 02:32:16.596074 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.596035    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmwvd\" (UniqueName: \"kubernetes.io/projected/a38673a0-5f73-4b57-8413-b960affe36d4-kube-api-access-mmwvd\") pod \"test-curl-7229-140527973828480\" (UID: \"a38673a0-5f73-4b57-8413-b960affe36d4\") " pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:16.596236 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.596111    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g62qq\" (UniqueName: \"kubernetes.io/projected/cae76225-9527-4aed-a527-76d938b576da-kube-api-access-g62qq\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:16.697107 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.697071    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmwvd\" (UniqueName: \"kubernetes.io/projected/a38673a0-5f73-4b57-8413-b960affe36d4-kube-api-access-mmwvd\") pod \"test-curl-7229-140527973828480\" (UID: \"a38673a0-5f73-4b57-8413-b960affe36d4\") " pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:16.705307 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.705274    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmwvd\" (UniqueName: \"kubernetes.io/projected/a38673a0-5f73-4b57-8413-b960affe36d4-kube-api-access-mmwvd\") pod \"test-curl-7229-140527973828480\" (UID: \"a38673a0-5f73-4b57-8413-b960affe36d4\") " pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:16.837062 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.836967    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:16.992542 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:16.992513    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7229-140527973828480"]
Jul 18 02:32:16.994521 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:32:16.994493    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda38673a0_5f73_4b57_8413_b960affe36d4.slice/crio-6ad2697b192b7bf488803102b8b02ea13b4f0b062567f276291df971cf18f3d1 WatchSource:0}: Error finding container 6ad2697b192b7bf488803102b8b02ea13b4f0b062567f276291df971cf18f3d1: Status 404 returned error can't find the container with id 6ad2697b192b7bf488803102b8b02ea13b4f0b062567f276291df971cf18f3d1
Jul 18 02:32:17.076939 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:17.076905    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cae76225-9527-4aed-a527-76d938b576da" path="/var/lib/kubelet/pods/cae76225-9527-4aed-a527-76d938b576da/volumes"
Jul 18 02:32:17.077381 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:17.077361    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1831732-c73e-426f-bb93-60dff070a65f" path="/var/lib/kubelet/pods/f1831732-c73e-426f-bb93-60dff070a65f/volumes"
Jul 18 02:32:17.215217 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:17.215185    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973828480" event={"ID":"a38673a0-5f73-4b57-8413-b960affe36d4","Type":"ContainerStarted","Data":"6ad2697b192b7bf488803102b8b02ea13b4f0b062567f276291df971cf18f3d1"}
Jul 18 02:32:17.216517 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:17.216499    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:17.216647 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:17.216507    2552 scope.go:117] "RemoveContainer" containerID="4ae25e648393d86c62290722e9f29f020948ca760e11afc5bd3d74de13a5bc98"
Jul 18 02:32:18.223917 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:18.223882    2552 generic.go:356] "Generic (PLEG): container finished" podID="a38673a0-5f73-4b57-8413-b960affe36d4" containerID="db07d9d6c20cd320935bcc20f1c5e0f2e1059884b983dfd2928e374980207b95" exitCode=0
Jul 18 02:32:18.224405 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:18.223976    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7229-140527973828480" event={"ID":"a38673a0-5f73-4b57-8413-b960affe36d4","Type":"ContainerDied","Data":"db07d9d6c20cd320935bcc20f1c5e0f2e1059884b983dfd2928e374980207b95"}
Jul 18 02:32:19.370632 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:19.370603    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:19.390303 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:19.390277    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7229-140527973828480_a38673a0-5f73-4b57-8413-b960affe36d4/test-curl-7229-140527973828480/0.log"
Jul 18 02:32:19.415677 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:19.415643    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7229-140527973828480"]
Jul 18 02:32:19.417987 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:19.417963    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7229-140527973828480"]
Jul 18 02:32:19.525087 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:19.525002    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmwvd\" (UniqueName: \"kubernetes.io/projected/a38673a0-5f73-4b57-8413-b960affe36d4-kube-api-access-mmwvd\") pod \"a38673a0-5f73-4b57-8413-b960affe36d4\" (UID: \"a38673a0-5f73-4b57-8413-b960affe36d4\") "
Jul 18 02:32:19.527236 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:19.527206    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a38673a0-5f73-4b57-8413-b960affe36d4-kube-api-access-mmwvd" (OuterVolumeSpecName: "kube-api-access-mmwvd") pod "a38673a0-5f73-4b57-8413-b960affe36d4" (UID: "a38673a0-5f73-4b57-8413-b960affe36d4"). InnerVolumeSpecName "kube-api-access-mmwvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:32:19.626030 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:19.625988    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmwvd\" (UniqueName: \"kubernetes.io/projected/a38673a0-5f73-4b57-8413-b960affe36d4-kube-api-access-mmwvd\") on node \"ip-10-0-128-48.ec2.internal\" DevicePath \"\""
Jul 18 02:32:20.121167 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:20.121132    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:20.221162 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:20.221123    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:20.237251 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:20.237220    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:20.237397 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:20.237250    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ad2697b192b7bf488803102b8b02ea13b4f0b062567f276291df971cf18f3d1"
Jul 18 02:32:20.246392 ip-10-0-128-48 kubenswrapper[2552]: E0718 02:32:20.246365    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7229-140527973828480\" is forbidden: User \"system:node:ip-10-0-128-48.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-48.ec2.internal' and this object" podUID="a38673a0-5f73-4b57-8413-b960affe36d4" pod="opendatahub/test-curl-7229-140527973828480"
Jul 18 02:32:21.076671 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:21.076639    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a38673a0-5f73-4b57-8413-b960affe36d4" path="/var/lib/kubelet/pods/a38673a0-5f73-4b57-8413-b960affe36d4/volumes"
Jul 18 02:32:30.204559 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:30.204526    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:33.451409 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:33.451374    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:33.553666 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:33.553639    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:34.679758 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:34.679717    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:35.787234 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:35.787176    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:36.969838 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:36.969803    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:37.069317 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:37.069286    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:41.311937 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:41.311906    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:41.412593 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:41.412559    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:41.604475 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:41.604360    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:41.842620 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:41.842589    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:42.470270 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:42.470238    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:42.571073 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:42.571044    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:52.390480 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:52.390447    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:57.038085 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:57.038051    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:32:57.442584 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:32:57.442548    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:02.182108 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:02.182075    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:03.620663 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:03.620626    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:03.675980 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:03.675944    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:03.913514 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:03.913485    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:04.135148 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:04.135094    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:10.167106 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:10.167074    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-v97ql_378c1a2d-982f-46ce-a506-88f8b567ccbd/manager/0.log"
Jul 18 02:33:10.285144 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:10.285113    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-spg4m_7d8b250e-4567-461b-abca-397c2ad014b4/manager/0.log"
Jul 18 02:33:10.403561 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:10.403523    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d774b7794-qksdj_e186f0cc-8eb2-41d7-9012-563752dff31a/manager/0.log"
Jul 18 02:33:10.521372 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:10.521290    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-szsqv_7e3b4a90-43a1-452c-b74a-1e77fc21a067/server/0.log"
Jul 18 02:33:10.637220 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:10.637192    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qcsz8_e37d7d55-2e3b-4cd2-8c2a-fe5078acd475/manager/0.log"
Jul 18 02:33:10.758410 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:10.758376    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b7456dd48-52dcc_1e5c1c18-3295-4685-b91a-ecbd7c97f4fa/copy-manifests/0.log"
Jul 18 02:33:10.772360 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:10.772282    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b7456dd48-52dcc_1e5c1c18-3295-4685-b91a-ecbd7c97f4fa/manager/0.log"
Jul 18 02:33:11.377975 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:11.377948    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6447d79b77-98lrf_79dd6c14-c2f5-4d6a-9188-f7e3530ee9b1/maas-api/0.log"
Jul 18 02:33:11.606716 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:11.606679    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29739015-9flw5_8c53420d-3de7-41b2-a1f9-9eaa13637dcf/cleanup/0.log"
Jul 18 02:33:11.831777 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:11.831745    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8k7nq_d8724144-3a41-4f5c-82f2-adb058bac1f6/postgres/0.log"
Jul 18 02:33:12.610981 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:12.610949    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn_0c33b59c-9f76-4033-88ca-9a2eab6b09ee/util/0.log"
Jul 18 02:33:12.617874 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:12.617849    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn_0c33b59c-9f76-4033-88ca-9a2eab6b09ee/pull/0.log"
Jul 18 02:33:12.623926 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:12.623910    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn_0c33b59c-9f76-4033-88ca-9a2eab6b09ee/extract/0.log"
Jul 18 02:33:12.739474 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:12.739417    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f_ea3b3415-162b-40bb-84e2-27afe590196b/util/0.log"
Jul 18 02:33:12.746638 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:12.746618    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f_ea3b3415-162b-40bb-84e2-27afe590196b/pull/0.log"
Jul 18 02:33:12.753331 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:12.753312    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f_ea3b3415-162b-40bb-84e2-27afe590196b/extract/0.log"
Jul 18 02:33:12.869887 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:12.869809    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz_9a0680e2-b156-4327-ae19-2ef7c9c508dd/util/0.log"
Jul 18 02:33:12.876557 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:12.876531    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz_9a0680e2-b156-4327-ae19-2ef7c9c508dd/pull/0.log"
Jul 18 02:33:12.883170 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:12.883153    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz_9a0680e2-b156-4327-ae19-2ef7c9c508dd/extract/0.log"
Jul 18 02:33:13.001142 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:13.001111    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk_a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1/extract/0.log"
Jul 18 02:33:13.007310 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:13.007289    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk_a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1/util/0.log"
Jul 18 02:33:13.013311 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:13.013287    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk_a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1/pull/0.log"
Jul 18 02:33:13.247706 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:13.247677    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5q2vd_db37bfdb-41b4-4e6c-8b15-f15f709318d6/manager/0.log"
Jul 18 02:33:13.365557 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:13.365528    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s95xl_4fec861d-14a1-4f83-be98-f0f632875348/manager/0.log"
Jul 18 02:33:13.479033 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:13.479001    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-b9qrq_bbb6acd5-eb2e-4147-a131-bbb17b336305/kuadrant-console-plugin/0.log"
Jul 18 02:33:13.598566 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:13.598488    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-slzb6_de2671e7-b13e-4801-af5b-a66419d98590/registry-server/0.log"
Jul 18 02:33:13.719980 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:13.719947    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-5jndx_537638b4-9cf1-4748-8d4a-b194fee80bd1/manager/0.log"
Jul 18 02:33:13.954123 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:13.954095    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-254h4_5efce080-fe0d-4a6b-b834-b2993e4a6d53/manager/0.log"
Jul 18 02:33:14.542290 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:14.542255    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f5976ff9-lftlr_0bfa6dcf-1126-4847-a901-92d8b90d866f/kube-auth-proxy/0.log"
Jul 18 02:33:14.888844 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:14.888765    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-rjql6_ea035ab2-c88a-4a1b-bc9b-826b51ee9474/payload-pre-processing/0.log"
Jul 18 02:33:15.005970 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:15.005935    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-5lgnb_fb3ee220-0029-4f10-baef-0a67a9893fd8/payload-processing/0.log"
Jul 18 02:33:15.117463 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:15.117375    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c89c86f68-22ccf_93ed0884-1025-4a1c-b573-dfa09a741a37/router/0.log"
Jul 18 02:33:15.275036 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:15.275012    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:33:15.285556 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:15.285536    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:33:15.575760 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:15.575737    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-c889s_65f40363-9b2c-483a-8d8e-5fad3f554e3b/main/0.log"
Jul 18 02:33:15.804118 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:15.804091    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8lgp_288ccfb5-0784-4828-ae02-9ece0c94864b/main/0.log"
Jul 18 02:33:21.061682 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.061624    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9d7x6/must-gather-n8rzs"]
Jul 18 02:33:21.062268 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.062194    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a38673a0-5f73-4b57-8413-b960affe36d4" containerName="test-curl-7229-140527973828480"
Jul 18 02:33:21.062268 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.062213    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a38673a0-5f73-4b57-8413-b960affe36d4" containerName="test-curl-7229-140527973828480"
Jul 18 02:33:21.062387 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.062305    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a38673a0-5f73-4b57-8413-b960affe36d4" containerName="test-curl-7229-140527973828480"
Jul 18 02:33:21.067078 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.067058    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9d7x6/must-gather-n8rzs"
Jul 18 02:33:21.069321 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.069296    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9d7x6\"/\"default-dockercfg-6b44b\""
Jul 18 02:33:21.069321 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.069320    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9d7x6\"/\"kube-root-ca.crt\""
Jul 18 02:33:21.069558 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.069335    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9d7x6\"/\"openshift-service-ca.crt\""
Jul 18 02:33:21.079365 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.079333    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8e44fcbd-1bab-4250-9b69-527fb003bc79-must-gather-output\") pod \"must-gather-n8rzs\" (UID: \"8e44fcbd-1bab-4250-9b69-527fb003bc79\") " pod="openshift-must-gather-9d7x6/must-gather-n8rzs"
Jul 18 02:33:21.079545 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.079382    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8cs2\" (UniqueName: \"kubernetes.io/projected/8e44fcbd-1bab-4250-9b69-527fb003bc79-kube-api-access-g8cs2\") pod \"must-gather-n8rzs\" (UID: \"8e44fcbd-1bab-4250-9b69-527fb003bc79\") " pod="openshift-must-gather-9d7x6/must-gather-n8rzs"
Jul 18 02:33:21.082490 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.082469    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9d7x6/must-gather-n8rzs"]
Jul 18 02:33:21.103006 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.102968    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:21.179866 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.179823    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8e44fcbd-1bab-4250-9b69-527fb003bc79-must-gather-output\") pod \"must-gather-n8rzs\" (UID: \"8e44fcbd-1bab-4250-9b69-527fb003bc79\") " pod="openshift-must-gather-9d7x6/must-gather-n8rzs"
Jul 18 02:33:21.179866 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.179869    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8cs2\" (UniqueName: \"kubernetes.io/projected/8e44fcbd-1bab-4250-9b69-527fb003bc79-kube-api-access-g8cs2\") pod \"must-gather-n8rzs\" (UID: \"8e44fcbd-1bab-4250-9b69-527fb003bc79\") " pod="openshift-must-gather-9d7x6/must-gather-n8rzs"
Jul 18 02:33:21.180187 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.180168    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8e44fcbd-1bab-4250-9b69-527fb003bc79-must-gather-output\") pod \"must-gather-n8rzs\" (UID: \"8e44fcbd-1bab-4250-9b69-527fb003bc79\") " pod="openshift-must-gather-9d7x6/must-gather-n8rzs"
Jul 18 02:33:21.187951 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.187920    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8cs2\" (UniqueName: \"kubernetes.io/projected/8e44fcbd-1bab-4250-9b69-527fb003bc79-kube-api-access-g8cs2\") pod \"must-gather-n8rzs\" (UID: \"8e44fcbd-1bab-4250-9b69-527fb003bc79\") " pod="openshift-must-gather-9d7x6/must-gather-n8rzs"
Jul 18 02:33:21.204064 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.204039    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:21.378023 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.377937    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9d7x6/must-gather-n8rzs"
Jul 18 02:33:21.527075 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.527048    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9d7x6/must-gather-n8rzs"]
Jul 18 02:33:21.528721 ip-10-0-128-48 kubenswrapper[2552]: W0718 02:33:21.528695    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e44fcbd_1bab_4250_9b69_527fb003bc79.slice/crio-8e5be0ad1e4efb67d79cf22db7d93f315f3ee168361a871515e481bba5e63536 WatchSource:0}: Error finding container 8e5be0ad1e4efb67d79cf22db7d93f315f3ee168361a871515e481bba5e63536: Status 404 returned error can't find the container with id 8e5be0ad1e4efb67d79cf22db7d93f315f3ee168361a871515e481bba5e63536
Jul 18 02:33:21.600891 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:21.600853    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9d7x6/must-gather-n8rzs" event={"ID":"8e44fcbd-1bab-4250-9b69-527fb003bc79","Type":"ContainerStarted","Data":"8e5be0ad1e4efb67d79cf22db7d93f315f3ee168361a871515e481bba5e63536"}
Jul 18 02:33:22.610014 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:22.609887    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9d7x6/must-gather-n8rzs" event={"ID":"8e44fcbd-1bab-4250-9b69-527fb003bc79","Type":"ContainerStarted","Data":"3e83f8b9b8c859da1b7c04dfe35782bb463f12615d47b719180915bea90a0ea1"}
Jul 18 02:33:22.610014 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:22.609944    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9d7x6/must-gather-n8rzs" event={"ID":"8e44fcbd-1bab-4250-9b69-527fb003bc79","Type":"ContainerStarted","Data":"5f6de136c412122137c03edb0d84805a90060740793f25c88843b62400a441c6"}
Jul 18 02:33:22.625975 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:22.625910    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9d7x6/must-gather-n8rzs" podStartSLOduration=0.849363503 podStartE2EDuration="1.625887747s" totalImagesPullingTime="776.524244ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:33:22.624630679 +0000 UTC m=+2708.064127614" watchObservedRunningTime="2026-07-18 02:33:22.625887747 +0000 UTC m=+2708.065384712"
Jul 18 02:33:23.931895 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:23.931863    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zr5mn_45860c98-3146-470d-8c2c-3feb65e32e10/global-pull-secret-syncer/0.log"
Jul 18 02:33:23.982648 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:23.982607    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log"
Jul 18 02:33:24.082308 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:24.082276    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-48.ec2.internal_099b81222a3a57c11d9d757e4965a26f/haproxy/0.log"
Jul 18 02:33:29.012785 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.012756    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn_0c33b59c-9f76-4033-88ca-9a2eab6b09ee/extract/0.log"
Jul 18 02:33:29.035511 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.035381    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn_0c33b59c-9f76-4033-88ca-9a2eab6b09ee/util/0.log"
Jul 18 02:33:29.057793 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.057742    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nvfsn_0c33b59c-9f76-4033-88ca-9a2eab6b09ee/pull/0.log"
Jul 18 02:33:29.083879 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.083853    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f_ea3b3415-162b-40bb-84e2-27afe590196b/extract/0.log"
Jul 18 02:33:29.104976 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.104950    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f_ea3b3415-162b-40bb-84e2-27afe590196b/util/0.log"
Jul 18 02:33:29.126522 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.126490    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lcq6f_ea3b3415-162b-40bb-84e2-27afe590196b/pull/0.log"
Jul 18 02:33:29.174923 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.174885    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz_9a0680e2-b156-4327-ae19-2ef7c9c508dd/extract/0.log"
Jul 18 02:33:29.200053 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.200015    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz_9a0680e2-b156-4327-ae19-2ef7c9c508dd/util/0.log"
Jul 18 02:33:29.224627 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.224597    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739spqz_9a0680e2-b156-4327-ae19-2ef7c9c508dd/pull/0.log"
Jul 18 02:33:29.252392 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.252359    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk_a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1/extract/0.log"
Jul 18 02:33:29.278729 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.278199    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk_a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1/util/0.log"
Jul 18 02:33:29.295573 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.295464    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnwxk_a9d7ee84-e2dc-4c36-ab7b-95085eeb39d1/pull/0.log"
Jul 18 02:33:29.676571 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.676516    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5q2vd_db37bfdb-41b4-4e6c-8b15-f15f709318d6/manager/0.log"
Jul 18 02:33:29.707667 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.707630    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s95xl_4fec861d-14a1-4f83-be98-f0f632875348/manager/0.log"
Jul 18 02:33:29.734031 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.734005    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-b9qrq_bbb6acd5-eb2e-4147-a131-bbb17b336305/kuadrant-console-plugin/0.log"
Jul 18 02:33:29.778882 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.778840    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-slzb6_de2671e7-b13e-4801-af5b-a66419d98590/registry-server/0.log"
Jul 18 02:33:29.937759 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:29.937598    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-5jndx_537638b4-9cf1-4748-8d4a-b194fee80bd1/manager/0.log"
Jul 18 02:33:30.041132 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:30.041056    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-254h4_5efce080-fe0d-4a6b-b834-b2993e4a6d53/manager/0.log"
Jul 18 02:33:31.901831 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:31.901791    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-m5m8z_46a417f1-eed7-4643-a01a-0cb5029dcc78/cluster-monitoring-operator/0.log"
Jul 18 02:33:32.155919 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:32.155833    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tzlxq_cecc90db-591c-451c-b527-bb1628faf2fc/node-exporter/0.log"
Jul 18 02:33:32.180318 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:32.180294    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tzlxq_cecc90db-591c-451c-b527-bb1628faf2fc/kube-rbac-proxy/0.log"
Jul 18 02:33:32.201514 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:32.201478    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tzlxq_cecc90db-591c-451c-b527-bb1628faf2fc/init-textfile/0.log"
Jul 18 02:33:33.966819 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:33.963824    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"]
Jul 18 02:33:33.974444 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:33.973554    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"]
Jul 18 02:33:33.974444 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:33.973677    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.018637 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.018534    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctff2\" (UniqueName: \"kubernetes.io/projected/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-kube-api-access-ctff2\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.018637 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.018610    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-sys\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.018898 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.018656    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-lib-modules\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.018898 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.018696    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-proc\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.018898 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.018725    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-podres\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.028508 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.028464    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-hzgdx_7081f4df-c702-41ab-b078-d62252a5b207/networking-console-plugin/0.log"
Jul 18 02:33:34.119993 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.119949    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctff2\" (UniqueName: \"kubernetes.io/projected/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-kube-api-access-ctff2\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.120195 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.120017    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-sys\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.120195 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.120066    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-lib-modules\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.120195 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.120107    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-proc\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.120195 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.120129    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-podres\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.120195 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.120173    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-sys\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.120536 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.120280    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-podres\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.120536 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.120300    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-lib-modules\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.120536 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.120337    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-proc\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.129330 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.129271    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctff2\" (UniqueName: \"kubernetes.io/projected/b85ed5b2-d4e6-4208-8b90-30f0ff8d4939-kube-api-access-ctff2\") pod \"perf-node-gather-daemonset-dglbz\" (UID: \"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939\") " pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.301060 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.300976    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.505136 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.505068    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"]
Jul 18 02:33:34.660779 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.660712    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/1.log"
Jul 18 02:33:34.667853 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.667808    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-gmrn7_2cbad028-6f01-4a6c-941f-966af50c85c0/console-operator/2.log"
Jul 18 02:33:34.726400 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.726356    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz" event={"ID":"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939","Type":"ContainerStarted","Data":"ee76a868979b72598eadabbe67637c9aaa9f8504dc69c828d3425d1f04ed8033"}
Jul 18 02:33:34.726607 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.726454    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz" event={"ID":"b85ed5b2-d4e6-4208-8b90-30f0ff8d4939","Type":"ContainerStarted","Data":"fb32c8a83674e9d648d1cc151e295ae8d7da97c7556ad3bbd84596acd5abc52d"}
Jul 18 02:33:34.727286 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.727256    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:34.742615 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:34.742569    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz" podStartSLOduration=1.7425495309999999 podStartE2EDuration="1.742549531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:33:34.74228119 +0000 UTC m=+2720.181778121" watchObservedRunningTime="2026-07-18 02:33:34.742549531 +0000 UTC m=+2720.182046463"
Jul 18 02:33:35.724842 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:35.724818    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-qklh8_24d78e74-d41d-4b85-a543-922c74ee852a/volume-data-source-validator/0.log"
Jul 18 02:33:36.803833 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:36.803803    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dvgh9_874287f6-35db-4ecb-9213-64ef5ed15dfa/dns/0.log"
Jul 18 02:33:36.823143 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:36.823116    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dvgh9_874287f6-35db-4ecb-9213-64ef5ed15dfa/kube-rbac-proxy/0.log"
Jul 18 02:33:36.937191 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:36.937159    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-strgq_ab7b9b44-0deb-42a5-9bc6-26778ed68ee5/dns-node-resolver/0.log"
Jul 18 02:33:37.670623 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:37.670587    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nqcrm_c5ac9a0d-d093-4e02-a9b2-402bddadb797/node-ca/0.log"
Jul 18 02:33:38.764901 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:38.764873    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f5976ff9-lftlr_0bfa6dcf-1126-4847-a901-92d8b90d866f/kube-auth-proxy/0.log"
Jul 18 02:33:38.875851 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:38.875822    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-rjql6_ea035ab2-c88a-4a1b-bc9b-826b51ee9474/payload-pre-processing/0.log"
Jul 18 02:33:38.907286 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:38.907258    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-5lgnb_fb3ee220-0029-4f10-baef-0a67a9893fd8/payload-processing/0.log"
Jul 18 02:33:38.930267 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:38.930218    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c89c86f68-22ccf_93ed0884-1025-4a1c-b573-dfa09a741a37/router/0.log"
Jul 18 02:33:39.614750 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:39.614718    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bdbxm_1e601610-f330-493b-adf7-13534056efa2/serve-healthcheck-canary/0.log"
Jul 18 02:33:40.131884 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:40.131850    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-hqv46_1d553bb2-7d34-4365-98b0-7b2b90227719/insights-operator/0.log"
Jul 18 02:33:40.132341 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:40.132321    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-hqv46_1d553bb2-7d34-4365-98b0-7b2b90227719/insights-operator/1.log"
Jul 18 02:33:40.229344 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:40.229311    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k9jpv_77725796-3a90-4975-aaad-61067e30c8a3/kube-rbac-proxy/0.log"
Jul 18 02:33:40.250535 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:40.250502    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k9jpv_77725796-3a90-4975-aaad-61067e30c8a3/exporter/0.log"
Jul 18 02:33:40.271230 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:40.271207    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k9jpv_77725796-3a90-4975-aaad-61067e30c8a3/extractor/0.log"
Jul 18 02:33:41.752022 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:41.751980    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9d7x6/perf-node-gather-daemonset-dglbz"
Jul 18 02:33:42.414777 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:42.414750    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-v97ql_378c1a2d-982f-46ce-a506-88f8b567ccbd/manager/0.log"
Jul 18 02:33:42.655248 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:42.655210    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-spg4m_7d8b250e-4567-461b-abca-397c2ad014b4/manager/0.log"
Jul 18 02:33:42.838497 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:42.838347    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d774b7794-qksdj_e186f0cc-8eb2-41d7-9012-563752dff31a/manager/0.log"
Jul 18 02:33:42.859236 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:42.859191    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-szsqv_7e3b4a90-43a1-452c-b74a-1e77fc21a067/server/0.log"
Jul 18 02:33:42.945792 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:42.945761    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qcsz8_e37d7d55-2e3b-4cd2-8c2a-fe5078acd475/manager/0.log"
Jul 18 02:33:43.055030 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:43.054998    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b7456dd48-52dcc_1e5c1c18-3295-4685-b91a-ecbd7c97f4fa/manager/0.log"
Jul 18 02:33:43.078869 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:43.078840    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b7456dd48-52dcc_1e5c1c18-3295-4685-b91a-ecbd7c97f4fa/copy-manifests/0.log"
Jul 18 02:33:44.859829 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:44.859802    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f5976ff9-lftlr_0bfa6dcf-1126-4847-a901-92d8b90d866f/kube-auth-proxy/0.log"
Jul 18 02:33:44.946783 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:44.946754    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-rjql6_ea035ab2-c88a-4a1b-bc9b-826b51ee9474/payload-pre-processing/0.log"
Jul 18 02:33:44.972079 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:44.972038    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-5lgnb_fb3ee220-0029-4f10-baef-0a67a9893fd8/payload-processing/0.log"
Jul 18 02:33:44.988716 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:44.988684    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c89c86f68-22ccf_93ed0884-1025-4a1c-b573-dfa09a741a37/router/0.log"
Jul 18 02:33:46.255410 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:46.255380    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dmwh7_97cd8d7f-4245-4fad-984a-e68481681a9e/konnectivity-agent/0.log" limit=1024
Jul 18 02:33:46.332877 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:46.332848    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f5976ff9-lftlr_0bfa6dcf-1126-4847-a901-92d8b90d866f/kube-auth-proxy/0.log"
Jul 18 02:33:46.427879 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:46.427850    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-rjql6_ea035ab2-c88a-4a1b-bc9b-826b51ee9474/payload-pre-processing/0.log"
Jul 18 02:33:46.450598 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:46.450528    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-5lgnb_fb3ee220-0029-4f10-baef-0a67a9893fd8/payload-processing/0.log"
Jul 18 02:33:46.463335 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:46.463277    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c89c86f68-22ccf_93ed0884-1025-4a1c-b573-dfa09a741a37/router/0.log"
Jul 18 02:33:49.838686 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:49.838630    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-7w6l2_31bfb9b1-86b3-4194-8651-68f48a251535/kube-storage-version-migrator-operator/1.log"
Jul 18 02:33:49.839780 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:49.839754    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-7w6l2_31bfb9b1-86b3-4194-8651-68f48a251535/kube-storage-version-migrator-operator/0.log"
Jul 18 02:33:50.841180 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:50.841146    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p4kc_422f305c-59b7-41b0-91bd-1fe60a29acda/kube-multus-additional-cni-plugins/0.log"
Jul 18 02:33:50.861450 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:50.861381    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p4kc_422f305c-59b7-41b0-91bd-1fe60a29acda/egress-router-binary-copy/0.log"
Jul 18 02:33:50.881156 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:50.881122    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p4kc_422f305c-59b7-41b0-91bd-1fe60a29acda/cni-plugins/0.log"
Jul 18 02:33:50.902232 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:50.902182    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p4kc_422f305c-59b7-41b0-91bd-1fe60a29acda/bond-cni-plugin/0.log"
Jul 18 02:33:50.921917 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:50.921892    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p4kc_422f305c-59b7-41b0-91bd-1fe60a29acda/routeoverride-cni/0.log"
Jul 18 02:33:50.944851 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:50.944830    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p4kc_422f305c-59b7-41b0-91bd-1fe60a29acda/whereabouts-cni-bincopy/0.log"
Jul 18 02:33:50.964502 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:50.964476    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p4kc_422f305c-59b7-41b0-91bd-1fe60a29acda/whereabouts-cni/0.log"
Jul 18 02:33:51.300946 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:51.300909    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-kwdzd_8e638ffe-a9f0-453f-b92c-d9d3ee930270/kube-multus/0.log"
Jul 18 02:33:51.375043 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:51.375016    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fgqqf_4de548f6-694f-425d-9ee4-5ac3df26aed4/network-metrics-daemon/0.log"
Jul 18 02:33:51.394404 ip-10-0-128-48 kubenswrapper[2552]: I0718 02:33:51.394371    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fgqqf_4de548f6-694f-425d-9ee4-5ac3df26aed4/kube-rbac-proxy/0.log"