Jul 09 03:40:50.509214 ip-10-0-134-164 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 03:40:50.509229 ip-10-0-134-164 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 03:40:50.509241 ip-10-0-134-164 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 03:40:50.509662 ip-10-0-134-164 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 03:41:00.679020 ip-10-0-134-164 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 09 03:41:00.679045 ip-10-0-134-164 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 30643d3c551b400fa7d323de02561340 --
Jul 09 03:43:03.063200 ip-10-0-134-164 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 03:43:03.439200 ip-10-0-134-164 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 03:43:03.439200 ip-10-0-134-164 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 03:43:03.439200 ip-10-0-134-164 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 03:43:03.439200 ip-10-0-134-164 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 03:43:03.439200 ip-10-0-134-164 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 03:43:03.440512 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.440413    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 03:43:03.445373 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445351    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 03:43:03.445373 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445371    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 03:43:03.445373 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445375    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 03:43:03.445373 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445378    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 03:43:03.445373 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445381    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445385    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445389    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445392    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445395    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445398    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445401    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445411    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445414    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445417    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445419    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445422    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445425    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445427    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445431    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445434    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445436    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445440    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445442    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445445    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 03:43:03.445568 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445448    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445450    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445453    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445456    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445458    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445461    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445465    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445469    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445472    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445475    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445478    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445481    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445484    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445487    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445490    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445494    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445497    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445499    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445503    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445505    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 03:43:03.446053 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445508    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445511    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445513    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445518    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445522    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445524    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445527    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445530    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445533    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445535    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445538    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445540    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445543    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445546    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445548    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445551    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445553    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445556    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445558    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 03:43:03.446579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445561    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445564    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445566    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445569    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445571    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445574    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445577    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445580    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445583    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445586    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445590    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445592    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445595    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445597    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445601    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445604    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445606    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445609    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445611    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 03:43:03.447042 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445615    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445618    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445621    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445625    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445628    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445630    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445633    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445636    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445639    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445642    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445648    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445650    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445653    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445656    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445658    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445661    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 03:43:03.447489 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.445663    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447703    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447711    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447714    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447717    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447720    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447724    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447727    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447730    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447732    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447735    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447737    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447740    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447743    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447746    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447748    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447751    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447754    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447756    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447759    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 03:43:03.447870 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447761    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447764    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447766    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447769    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447772    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447777    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447780    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447783    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447786    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447789    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447792    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447794    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447797    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447800    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447804    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447807    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447810    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447813    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447815    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447818    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 03:43:03.448366 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447821    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447824    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447826    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447829    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447831    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447834    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447837    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447839    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447842    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447844    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447847    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447849    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447852    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447854    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447857    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447876    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447880    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447883    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447886    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 03:43:03.448859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447889    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447891    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447894    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447897    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447901    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447905    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447908    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447911    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447914    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447918    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447922    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447925    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447928    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447932    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447935    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447938    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447940    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447943    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447946    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 03:43:03.449338 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447948    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447951    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447954    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447956    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447959    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447961    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447964    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447966    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447969    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447971    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447974    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447977    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447980    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447982    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447985    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447988    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447991    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447993    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447996    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.447999    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 03:43:03.449798 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448002    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448005    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448387    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448393    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448396    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448398    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448402    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448405    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448408    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448410    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448413    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448416    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448418    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448421    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448424    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448426    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448429    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448432    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448434    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448437    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 03:43:03.450288 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448439    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448442    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448444    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448447    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448450    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448453    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448458    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448461    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448463    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448466    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448468    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448471    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448474    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448477    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448479    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448482    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448485    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448488    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448490    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448493    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 03:43:03.450808 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448496    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448498    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448501    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448504    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448508    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448510    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448513    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448516    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448519    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448522    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448525    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448527    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448529    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448532    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448535    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448538    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448541    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448543    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448546    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 03:43:03.451294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448549    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448552    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448555    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448557    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448560    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448563    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448565    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448568    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448571    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448573    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448576    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448578    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448583    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448588    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448591    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448594    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448597    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448600    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448602    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448605    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 03:43:03.451763 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448608    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448610    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448613    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448615    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448618    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448620    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448622    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448625    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448628    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448631    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448633    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448636    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448639    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448642    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448644    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448648    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448652    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448655    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448658    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448661    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 03:43:03.452257 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448663    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.448666    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448749    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448756    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448762    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448767    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448773    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448777    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448782    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448786    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448790    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448793    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448796    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448800    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448803    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448806    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448809    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448812    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448815    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448818    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448823    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448827    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448830    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448833    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 03:43:03.452733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448837    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448841    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448844    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448847    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448851    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448854    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448857    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448877    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448881    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448884    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448888    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448891    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448894    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448897    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448900    2582 flags.go:64] FLAG: --enable-server="true"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448909    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448914    2582 flags.go:64] FLAG: --event-burst="100"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448917    2582 flags.go:64] FLAG: --event-qps="50"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448920    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448924    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448927    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448935    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448938    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448941    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448944    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 09 03:43:03.453329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448947    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448950    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448953    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448957    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448960    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448963    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448965    2582 flags.go:64] FLAG: --feature-gates=""
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448970    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448973    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448976    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448979    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448983    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448986    2582 flags.go:64] FLAG: --help="false"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448989    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448992    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448995    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.448998    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449001    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449005    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449008    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449011    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449014    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449017    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 03:43:03.453949 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449020    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449023    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449026    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449029    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449032    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449035    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449038    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449041    2582 flags.go:64] FLAG: --lock-file=""
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449043    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449046    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449049    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449055    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449060    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449063    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449066    2582 flags.go:64] FLAG: --logging-format="text"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449073    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449076    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449080    2582 flags.go:64] FLAG: --manifest-url=""
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449083    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449087    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449098    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449103    2582 flags.go:64] FLAG: --max-pods="110"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449106    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449109    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449112    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 03:43:03.454565 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449115    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449118    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449121    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449124    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449132    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449135    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449138    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449141    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449145    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449151    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449154    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449157    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449160    2582 flags.go:64] FLAG: --port="10250"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449164    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449167    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a8d74569c9977823"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449170    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449174    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449177    2582 flags.go:64] FLAG: --register-node="true"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449179    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449185    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449188    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449191    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449194    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449199    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449202    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 03:43:03.455186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449205    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449208    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449211    2582 flags.go:64] FLAG: --runonce="false"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449214    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449217    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449220    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449223    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449226    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449229    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449233    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449235    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449238    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449242    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449244    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449247    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449250    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449253    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449256    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449262    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449265    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449268    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.449999    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.450003    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.450006    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.450009    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 03:43:03.455802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.450013    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.450017    2582 flags.go:64] FLAG: --v="2"
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.450022    2582 flags.go:64] FLAG: --version="false"
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.450026    2582 flags.go:64] FLAG: --vmodule=""
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.450030    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.450035    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450177    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450181    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450184    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450187    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450190    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450193    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450196    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450198    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450201    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450204    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450206    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450209    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450212    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450215    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450217    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 03:43:03.456579 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450221    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450223    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450226    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450229    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450232    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450234    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450237    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450240    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450243    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450245    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450248    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450251    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450255    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450257    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450262    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450265    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450270    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450273    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450275    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 03:43:03.457114 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450278    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450281    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450284    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450287    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450290    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450293    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450296    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450299    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450302    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450304    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450307    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450310    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450312    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450315    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450317    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450320    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450323    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450326    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450329    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450331    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 03:43:03.457592 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450334    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450337    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450339    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450342    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450344    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450349    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450352    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450354    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450357    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450361    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450364    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450368    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450371    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450374    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450379    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450383    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450387    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450389    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450392    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 03:43:03.458098 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450395    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450398    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450400    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450403    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450406    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450408    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450411    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450414    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450416    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450419    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450422    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450425    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450427    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450430    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450433    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450435    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450438    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450440    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450444    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450447    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 03:43:03.458578 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450449    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450452    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450457    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450460    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450462    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.450465    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.450470    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.457362    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.457380    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.457398    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.457404    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457462    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457467    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457470    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457473    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 03:43:03.459091 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457476    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457479    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457483    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457487    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457491    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457494    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457497    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457501    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457505    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457507    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457510    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457513    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457516    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457518    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457521    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457523    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457526    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457529    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457531    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 03:43:03.459470 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457534    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457538    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457540    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457543    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457545    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457548    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457552    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457555    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457558    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457561    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457565    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457567    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457570    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457574    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457578    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457580    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457583    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457586    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457588    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 03:43:03.459974 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457592    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457595    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457599    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457604    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457607    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457611    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457614    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457617    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457619    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457622    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457624    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457627    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457630    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457633    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457635    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457638    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457640    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457643    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457645    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457649    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 03:43:03.460442 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457652    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457655    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457658    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457660    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457663    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457666    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457668    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457671    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457674    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457676    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457679    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457681    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457684    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457687    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457690    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457692    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457696    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457698    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457701    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457703    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 03:43:03.460990 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457706    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457708    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457711    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457714    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457716    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457719    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457722    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457724    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457727    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457730    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457732    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457736    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457739    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457741    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457744    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457747    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 03:43:03.461499 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457749    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.457754    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457891    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457896    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457899    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457903    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457906    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457908    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457911    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457914    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457916    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457919    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457921    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457925    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457927    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 03:43:03.461906 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457930    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457932    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457935    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457938    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457940    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457943    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457946    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457948    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457951    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457954    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457956    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457959    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457962    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457964    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457968    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457972    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457976    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457979    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457981    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457984    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 03:43:03.462303 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457987    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457990    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457992    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457995    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.457997    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458000    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458002    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458005    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458007    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458010    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458013    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458019    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458022    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458024    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458027    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458030    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458032    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458035    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458038    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458040    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 03:43:03.462787 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458043    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458046    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458048    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458051    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458054    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458056    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458059    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458063    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458066    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458069    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458071    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458074    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458077    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458079    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458082    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458085    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458087    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458090    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458092    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458095    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 03:43:03.463376 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458097    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458100    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458103    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458105    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458109    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458111    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458114    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458117    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458119    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458122    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458124    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458127    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458130    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458132    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458135    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458138    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458141    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458144    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458146    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458149    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 03:43:03.463859 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458151    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 03:43:03.464345 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458154    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 03:43:03.464345 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458157    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 03:43:03.464345 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458159    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 03:43:03.464345 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458163    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 03:43:03.464345 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:03.458166    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 03:43:03.464345 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.458171    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 03:43:03.464345 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.458306    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 03:43:03.464345 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.460419    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 03:43:03.464345 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.461896    2582 server.go:1019] "Starting client certificate rotation"
Jul 09 03:43:03.464345 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.461996    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 03:43:03.464345 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.462029    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 03:43:03.465012 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.464996    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 03:43:03.470806 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.470778    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 03:43:03.486918 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.486888    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 09 03:43:03.489555 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.489536    2582 log.go:25] "Validated CRI v1 image API"
Jul 09 03:43:03.490457 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.490441    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 03:43:03.493622 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.493601    2582 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a0b097f5-b922-4d24-98fe-43cbc4177c68:/dev/nvme0n1p3 f090172b-403e-4b60-b206-c5a761952017:/dev/nvme0n1p4]
Jul 09 03:43:03.493673 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.493621    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 03:43:03.494953 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.494928    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 03:43:03.498885 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.498758    2582 manager.go:216] Machine: {Timestamp:2026-07-09 03:43:03.49757996 +0000 UTC m=+0.316835729 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091194 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cbcc793786ad60a5bbb2e83752af6 SystemUUID:ec2cbcc7-9378-6ad6-0a5b-bb2e83752af6 BootID:30643d3c-551b-400f-a7d3-23de02561340 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:eb:a3:9d:c5:07 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:eb:a3:9d:c5:07 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:34:47:22:58:20 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 03:43:03.498981 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.498886    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 03:43:03.499012 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.498989    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 03:43:03.501955 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.501914    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 03:43:03.502119 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.501958    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-164.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 03:43:03.502678 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.502667    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 03:43:03.502713 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.502680    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 03:43:03.502713 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.502688    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 03:43:03.503386 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.503374    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 03:43:03.504738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.504726    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 03:43:03.508929 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.508913    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 03:43:03.509069 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.509058    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 03:43:03.509111 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.509102    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 03:43:03.509150 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.509128    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 03:43:03.509179 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.509156    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 03:43:03.509179 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.509176    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 09 03:43:03.509233 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.509194    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 03:43:03.509675 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.509662    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 03:43:03.509746 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.509735    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 03:43:03.511907 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.511886    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 09 03:43:03.514004 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.513989    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 03:43:03.514056 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.514022    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 03:43:03.515277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515264    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 03:43:03.515338 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515281    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 03:43:03.515338 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515287    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 03:43:03.515338 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515293    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 03:43:03.515338 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515302    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 03:43:03.515338 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515309    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 03:43:03.515338 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515321    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 03:43:03.515338 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515327    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 03:43:03.515338 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515334    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 03:43:03.515338 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515340    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 03:43:03.515571 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515354    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 03:43:03.515571 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.515362    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 03:43:03.516145 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.516134    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 03:43:03.516145 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.516145    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 03:43:03.516288 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.516270    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bphmt"
Jul 09 03:43:03.520363 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.520323    2582 server.go:1268] "Started kubelet"
Jul 09 03:43:03.520484 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.520435    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 03:43:03.520547 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.520529    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 03:43:03.520643 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.520605    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 03:43:03.521253 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.521213    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 03:43:03.521351 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.521254    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:03.521405 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.521389    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 03:43:03.521694 ip-10-0-134-164 systemd[1]: Started Kubernetes Kubelet.
Jul 09 03:43:03.524064 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.524035    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 03:43:03.524776 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.524760    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 03:43:03.524938 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.524894    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 03:43:03.530588 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.530569    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 03:43:03.530667 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.530573    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 03:43:03.530716 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.530670    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-164.ec2.internal\" not found"
Jul 09 03:43:03.530992 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.529798    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4d45017f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.520223615 +0000 UTC m=+0.339479387,LastTimestamp:2026-07-09 03:43:03.520223615 +0000 UTC m=+0.339479387,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.531510 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531492    2582 factory.go:55] Registering systemd factory
Jul 09 03:43:03.531628 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531514    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 09 03:43:03.531628 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531594    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 03:43:03.531628 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531605    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 03:43:03.531749 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531701    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 03:43:03.531749 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531712    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 03:43:03.531749 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531741    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 03:43:03.531891 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531772    2582 factory.go:153] Registering CRI-O factory
Jul 09 03:43:03.531891 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531787    2582 factory.go:223] Registration of the crio container factory successfully
Jul 09 03:43:03.531891 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531843    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 03:43:03.531891 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531887    2582 factory.go:103] Registering Raw factory
Jul 09 03:43:03.532038 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.531915    2582 manager.go:1196] Started watching for new ooms in manager
Jul 09 03:43:03.532294 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.532283    2582 manager.go:349] Starting recovery of all containers
Jul 09 03:43:03.536167 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.536135    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 03:43:03.536263 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.536201    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 03:43:03.543459 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.543320    2582 manager.go:354] Recovery completed
Jul 09 03:43:03.550021 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.550003    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:03.552130 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.552110    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:03.552191 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.552152    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:03.552191 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.552167    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:03.552783 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.552766    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 03:43:03.552783 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.552779    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 03:43:03.552953 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.552799    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 03:43:03.552953 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.552819    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:03.553566 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.553548    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:03.553653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.553573    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:03.553653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.553585    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:03.554273 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.554203    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2be779  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552132985 +0000 UTC m=+0.371388764,LastTimestamp:2026-07-09 03:43:03.552132985 +0000 UTC m=+0.371388764,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.555152 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.555140    2582 policy_none.go:49] "None policy: Start"
Jul 09 03:43:03.555194 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.555157    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 03:43:03.555194 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.555168    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 03:43:03.557150 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.557135    2582 policy_none.go:47] "Start"
Jul 09 03:43:03.561619 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.561547    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552158984 +0000 UTC m=+0.371414757,LastTimestamp:2026-07-09 03:43:03.552158984 +0000 UTC m=+0.371414757,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.572155 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.572064    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c839d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552172957 +0000 UTC m=+0.371428728,LastTimestamp:2026-07-09 03:43:03.552172957 +0000 UTC m=+0.371428728,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.584464 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.584387    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2be779\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2be779  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552132985 +0000 UTC m=+0.371388764,LastTimestamp:2026-07-09 03:43:03.553558909 +0000 UTC m=+0.372814678,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.593413 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.593334    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552158984 +0000 UTC m=+0.371414757,LastTimestamp:2026-07-09 03:43:03.553579854 +0000 UTC m=+0.372835625,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.596618 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.596604    2582 manager.go:337] "Starting Device Plugin manager"
Jul 09 03:43:03.596692 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.596642    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 03:43:03.596692 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.596656    2582 server.go:87] "Starting device plugin registration server"
Jul 09 03:43:03.596970 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.596956    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 03:43:03.597036 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.596971    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 03:43:03.597083 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.597035    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 03:43:03.597159 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.597145    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 03:43:03.597210 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.597162    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 03:43:03.597780 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.597757    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 03:43:03.597855 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.597822    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-164.ec2.internal\" not found"
Jul 09 03:43:03.604137 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.604048    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c839d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c839d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552172957 +0000 UTC m=+0.371428728,LastTimestamp:2026-07-09 03:43:03.553589826 +0000 UTC m=+0.372845595,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.627711 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.627628    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d531f2004  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.618404356 +0000 UTC m=+0.437660111,LastTimestamp:2026-07-09 03:43:03.618404356 +0000 UTC m=+0.437660111,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.698122 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.698009    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:03.699382 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.699365    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:03.699500 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.699396    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:03.699500 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.699407    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:03.699500 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.699434    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.705932 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.705896    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 03:43:03.707424 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.707400    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 03:43:03.707519 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.707448    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 03:43:03.707519 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.707478    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 03:43:03.707519 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.707514    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 03:43:03.709929 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.709904    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 09 03:43:03.710045 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.709856    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2be779\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2be779  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552132985 +0000 UTC m=+0.371388764,LastTimestamp:2026-07-09 03:43:03.699381152 +0000 UTC m=+0.518636921,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.719197 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.719167    2582 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.719350 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.719282    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552158984 +0000 UTC m=+0.371414757,LastTimestamp:2026-07-09 03:43:03.699401747 +0000 UTC m=+0.518657516,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.728207 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.728135    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c839d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c839d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552172957 +0000 UTC m=+0.371428728,LastTimestamp:2026-07-09 03:43:03.699411522 +0000 UTC m=+0.518667291,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.745804 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.745772    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms"
Jul 09 03:43:03.808034 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.808000    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-164.ec2.internal"]
Jul 09 03:43:03.808115 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.808099    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:03.809028 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.809011    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:03.809137 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.809039    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:03.809137 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.809050    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:03.810967 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.810953    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:03.811102 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.811085    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.811141 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.811119    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:03.811662 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.811645    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:03.811752 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.811645    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:03.811752 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.811707    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:03.811752 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.811725    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:03.811752 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.811678    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:03.811954 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.811769    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:03.813447 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.813432    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.813524 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.813460    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:03.814132 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.814116    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:03.814222 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.814148    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:03.814222 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.814162    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:03.818397 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.818323    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2be779\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2be779  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552132985 +0000 UTC m=+0.371388764,LastTimestamp:2026-07-09 03:43:03.809025037 +0000 UTC m=+0.628280805,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.830470 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.830404    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552158984 +0000 UTC m=+0.371414757,LastTimestamp:2026-07-09 03:43:03.809043591 +0000 UTC m=+0.628299359,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.839235 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.839155    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c839d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c839d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552172957 +0000 UTC m=+0.371428728,LastTimestamp:2026-07-09 03:43:03.809056184 +0000 UTC m=+0.628311955,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.840918 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.840900    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-164.ec2.internal\" not found" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.845779 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.845759    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-164.ec2.internal\" not found" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.849969 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.849902    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2be779\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2be779  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552132985 +0000 UTC m=+0.371388764,LastTimestamp:2026-07-09 03:43:03.81166669 +0000 UTC m=+0.630922461,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.858802 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.858724    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2be779\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2be779  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552132985 +0000 UTC m=+0.371388764,LastTimestamp:2026-07-09 03:43:03.811689116 +0000 UTC m=+0.630944892,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.867819 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.867729    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552158984 +0000 UTC m=+0.371414757,LastTimestamp:2026-07-09 03:43:03.811716974 +0000 UTC m=+0.630972746,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.877804 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.877722    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c839d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c839d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552172957 +0000 UTC m=+0.371428728,LastTimestamp:2026-07-09 03:43:03.811731033 +0000 UTC m=+0.630986803,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.887130 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.887033    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552158984 +0000 UTC m=+0.371414757,LastTimestamp:2026-07-09 03:43:03.811750159 +0000 UTC m=+0.631005931,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.895664 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.895585    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c839d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c839d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552172957 +0000 UTC m=+0.371428728,LastTimestamp:2026-07-09 03:43:03.811775692 +0000 UTC m=+0.631031461,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.904520 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.904435    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2be779\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2be779  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552132985 +0000 UTC m=+0.371388764,LastTimestamp:2026-07-09 03:43:03.814130436 +0000 UTC m=+0.633386206,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.915076 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.914991    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552158984 +0000 UTC m=+0.371414757,LastTimestamp:2026-07-09 03:43:03.814154995 +0000 UTC m=+0.633410765,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.919974 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.919953    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:03.921285 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.921264    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:03.921388 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.921302    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:03.921388 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.921315    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:03.921388 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.921353    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.922965 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.922900    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c839d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c839d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552172957 +0000 UTC m=+0.371428728,LastTimestamp:2026-07-09 03:43:03.814167555 +0000 UTC m=+0.633423521,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.932118 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.932090    2582 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.932118 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.932031    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2be779\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2be779  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552132985 +0000 UTC m=+0.371388764,LastTimestamp:2026-07-09 03:43:03.921286342 +0000 UTC m=+0.740542113,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:03.932704 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.932685    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3353beadccb3550d362986f67bd77010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal\" (UID: \"3353beadccb3550d362986f67bd77010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.932775 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.932713    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3353beadccb3550d362986f67bd77010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal\" (UID: \"3353beadccb3550d362986f67bd77010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.932775 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:03.932730    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d922faf733bce56c232f28d751a7e10-config\") pod \"kube-apiserver-proxy-ip-10-0-134-164.ec2.internal\" (UID: \"3d922faf733bce56c232f28d751a7e10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:03.940301 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:03.940228    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-164.ec2.internal.18c0822d4f2c4d08  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-164.ec2.internal,UID:ip-10-0-134-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-164.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:03.552158984 +0000 UTC m=+0.371414757,LastTimestamp:2026-07-09 03:43:03.92130772 +0000 UTC m=+0.740563490,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:04.032921 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.032828    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3353beadccb3550d362986f67bd77010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal\" (UID: \"3353beadccb3550d362986f67bd77010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:04.033008 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.032916    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d922faf733bce56c232f28d751a7e10-config\") pod \"kube-apiserver-proxy-ip-10-0-134-164.ec2.internal\" (UID: \"3d922faf733bce56c232f28d751a7e10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:04.033008 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.032937    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3353beadccb3550d362986f67bd77010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal\" (UID: \"3353beadccb3550d362986f67bd77010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:04.033008 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.032947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3353beadccb3550d362986f67bd77010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal\" (UID: \"3353beadccb3550d362986f67bd77010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:04.033008 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.032992    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d922faf733bce56c232f28d751a7e10-config\") pod \"kube-apiserver-proxy-ip-10-0-134-164.ec2.internal\" (UID: \"3d922faf733bce56c232f28d751a7e10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:04.033008 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.032997    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3353beadccb3550d362986f67bd77010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal\" (UID: \"3353beadccb3550d362986f67bd77010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:04.143360 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.143318    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:04.148947 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.148928    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:04.155690 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:04.155668    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms"
Jul 09 03:43:04.332651 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.332556    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:04.333903 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.333882    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:04.334011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.333917    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:04.334011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.333927    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:04.334011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.333953    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:04.350656 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:04.350618    2582 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:04.473700 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:04.473665    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 03:43:04.516082 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:04.516048    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 03:43:04.530353 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.530326    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:04.656417 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.656379    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:04.657685 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.657660    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:04.657831 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.657696    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:04.657831 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.657710    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:04.770318 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:04.770282    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3353beadccb3550d362986f67bd77010.slice/crio-c4ac1d1a6bd61bea48c0bb98afc4e206cb8b06b9b53d70dfcca8befa83a6d8b9 WatchSource:0}: Error finding container c4ac1d1a6bd61bea48c0bb98afc4e206cb8b06b9b53d70dfcca8befa83a6d8b9: Status 404 returned error can't find the container with id c4ac1d1a6bd61bea48c0bb98afc4e206cb8b06b9b53d70dfcca8befa83a6d8b9
Jul 09 03:43:04.770501 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:04.770481    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d922faf733bce56c232f28d751a7e10.slice/crio-aa11f45035125f98304e99d7cca95456364ef3ba0edf78346ad707b082dd3c60 WatchSource:0}: Error finding container aa11f45035125f98304e99d7cca95456364ef3ba0edf78346ad707b082dd3c60: Status 404 returned error can't find the container with id aa11f45035125f98304e99d7cca95456364ef3ba0edf78346ad707b082dd3c60
Jul 09 03:43:04.776253 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:04.776237    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 03:43:04.785897 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:04.785798    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822d9824f4e3  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\",Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:04.776414435 +0000 UTC m=+1.595670191,LastTimestamp:2026-07-09 03:43:04.776414435 +0000 UTC m=+1.595670191,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:04.794492 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:04.794462    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 09 03:43:04.794625 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:04.794499    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-134-164.ec2.internal.18c0822d9825c355  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-134-164.ec2.internal,UID:3d922faf733bce56c232f28d751a7e10,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1\",Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:04.776467285 +0000 UTC m=+1.595723040,LastTimestamp:2026-07-09 03:43:04.776467285 +0000 UTC m=+1.595723040,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:04.965702 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:04.965609    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s"
Jul 09 03:43:05.091991 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:05.091955    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 03:43:05.151510 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:05.151477    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:05.152464 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:05.152441    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:05.152608 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:05.152479    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:05.152608 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:05.152494    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:05.152608 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:05.152524    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:05.168803 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:05.168770    2582 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:05.532152 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:05.532112    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:05.713218 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:05.713153    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal" event={"ID":"3353beadccb3550d362986f67bd77010","Type":"ContainerStarted","Data":"c4ac1d1a6bd61bea48c0bb98afc4e206cb8b06b9b53d70dfcca8befa83a6d8b9"}
Jul 09 03:43:05.714906 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:05.714853    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-164.ec2.internal" event={"ID":"3d922faf733bce56c232f28d751a7e10","Type":"ContainerStarted","Data":"aa11f45035125f98304e99d7cca95456364ef3ba0edf78346ad707b082dd3c60"}
Jul 09 03:43:06.460504 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:06.460461    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 03:43:06.530269 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:06.530235    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:06.577240 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:06.577200    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="3.2s"
Jul 09 03:43:06.769805 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:06.769717    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:06.770845 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:06.770824    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:06.770963 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:06.770859    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:06.770963 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:06.770886    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:06.770963 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:06.770921    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:06.789204 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:06.789170    2582 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:06.837124 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:06.837034    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e1263791d  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\" in 2.05s (2.05s including waiting). Image size: 303912755 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:06.827331869 +0000 UTC m=+3.646587646,LastTimestamp:2026-07-09 03:43:06.827331869 +0000 UTC m=+3.646587646,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:06.875746 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:06.875663    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-134-164.ec2.internal.18c0822e14bd8707  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-134-164.ec2.internal,UID:3d922faf733bce56c232f28d751a7e10,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1\" in 2.09s (2.09s including waiting). Image size: 342879910 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:06.866788103 +0000 UTC m=+3.686043858,LastTimestamp:2026-07-09 03:43:06.866788103 +0000 UTC m=+3.686043858,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:06.938487 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:06.938257    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-134-164.ec2.internal.18c0822e1874b968  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-134-164.ec2.internal,UID:3d922faf733bce56c232f28d751a7e10,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Created,Message:Created container: haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:06.929125736 +0000 UTC m=+3.748381513,LastTimestamp:2026-07-09 03:43:06.929125736 +0000 UTC m=+3.748381513,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:06.948067 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:06.947981    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-134-164.ec2.internal.18c0822e18ec222a  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-134-164.ec2.internal,UID:3d922faf733bce56c232f28d751a7e10,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Started,Message:Started container haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:06.936951338 +0000 UTC m=+3.756207110,LastTimestamp:2026-07-09 03:43:06.936951338 +0000 UTC m=+3.756207110,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:06.987351 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:06.987317    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:06.988331 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:06.988308    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:06.988462 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:06.988344    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:06.988462 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:06.988354    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:07.095782 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:07.095697    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e21bf5e0e  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:07.085012494 +0000 UTC m=+3.904268262,LastTimestamp:2026-07-09 03:43:07.085012494 +0000 UTC m=+3.904268262,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:07.104014 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:07.103860    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e22483031  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:07.093979185 +0000 UTC m=+3.913234963,LastTimestamp:2026-07-09 03:43:07.093979185 +0000 UTC m=+3.913234963,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:07.234203 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:07.234168    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 03:43:07.440472 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:07.440394    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 09 03:43:07.530282 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.530255    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:07.719700 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.719606    2582 generic.go:356] "Generic (PLEG): container finished" podID="3353beadccb3550d362986f67bd77010" containerID="956c7dc12cf1fabf522d9d6e0f1382d62f1204a06a81480331025ffba87a36f6" exitCode=0
Jul 09 03:43:07.719700 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.719678    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:07.720173 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.719692    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal" event={"ID":"3353beadccb3550d362986f67bd77010","Type":"ContainerDied","Data":"956c7dc12cf1fabf522d9d6e0f1382d62f1204a06a81480331025ffba87a36f6"}
Jul 09 03:43:07.720623 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.720600    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:07.720714 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.720638    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:07.720714 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.720653    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:07.720926 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:07.720910    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-164.ec2.internal\" not found" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:07.721222 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.721199    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-164.ec2.internal" event={"ID":"3d922faf733bce56c232f28d751a7e10","Type":"ContainerStarted","Data":"7321100bc82629bd52edb810676f21b79f6ed2b89d9e297a348cb3b3a69b8fc3"}
Jul 09 03:43:07.721281 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.721253    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:07.721938 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.721917    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:07.722052 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.721947    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:07.722052 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:07.721957    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:07.722154 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:07.722107    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-164.ec2.internal\" not found" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:07.732339 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:07.732260    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e47c534b7  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:07.722929335 +0000 UTC m=+4.542185091,LastTimestamp:2026-07-09 03:43:07.722929335 +0000 UTC m=+4.542185091,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:07.813240 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:07.813135    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e4c9d22ee  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Created,Message:Created container: kube-rbac-proxy-crio,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:07.804189422 +0000 UTC m=+4.623445199,LastTimestamp:2026-07-09 03:43:07.804189422 +0000 UTC m=+4.623445199,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:07.821935 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:07.821807    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e4d237b94  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Started,Message:Started container kube-rbac-proxy-crio,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:07.81299394 +0000 UTC m=+4.632249696,LastTimestamp:2026-07-09 03:43:07.81299394 +0000 UTC m=+4.632249696,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:07.956985 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:07.956953    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 03:43:08.531245 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.531215    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:08.724368 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.724342    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/0.log"
Jul 09 03:43:08.724765 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.724715    2582 generic.go:356] "Generic (PLEG): container finished" podID="3353beadccb3550d362986f67bd77010" containerID="b45f76b5d6cde2bdbffaf966722ec58f04c9a31adbdd23fe6196cbf858789678" exitCode=1
Jul 09 03:43:08.724822 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.724779    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:08.724822 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.724808    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:08.724822 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.724807    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal" event={"ID":"3353beadccb3550d362986f67bd77010","Type":"ContainerDied","Data":"b45f76b5d6cde2bdbffaf966722ec58f04c9a31adbdd23fe6196cbf858789678"}
Jul 09 03:43:08.725580 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.725565    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:08.725653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.725590    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:08.725653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.725600    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:08.725653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.725610    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:08.725653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.725628    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:08.725653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.725638    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:08.725839 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:08.725823    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-164.ec2.internal\" not found" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:08.725895 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:08.725848    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-164.ec2.internal\" not found" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:08.725932 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:08.725907    2582 scope.go:117] "RemoveContainer" containerID="b45f76b5d6cde2bdbffaf966722ec58f04c9a31adbdd23fe6196cbf858789678"
Jul 09 03:43:08.737089 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:08.736975    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e47c534b7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e47c534b7  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:07.722929335 +0000 UTC m=+4.542185091,LastTimestamp:2026-07-09 03:43:08.729074537 +0000 UTC m=+5.548330297,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:08.815958 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:08.815860    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e4c9d22ee\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e4c9d22ee  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Created,Message:Created container: kube-rbac-proxy-crio,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:07.804189422 +0000 UTC m=+4.623445199,LastTimestamp:2026-07-09 03:43:08.805784872 +0000 UTC m=+5.625040643,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:08.824744 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:08.824658    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e4d237b94\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822e4d237b94  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Started,Message:Started container kube-rbac-proxy-crio,Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:07.81299394 +0000 UTC m=+4.632249696,LastTimestamp:2026-07-09 03:43:08.814441153 +0000 UTC m=+5.633696914,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:09.529712 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.529680    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:09.729134 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.729064    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 03:43:09.729555 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.729538    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/0.log"
Jul 09 03:43:09.729914 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.729895    2582 generic.go:356] "Generic (PLEG): container finished" podID="3353beadccb3550d362986f67bd77010" containerID="6c01c8ff5b416646ec9344ec2389ba70503981e2fbcfd85601375c2a6289fb3d" exitCode=1
Jul 09 03:43:09.729961 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.729927    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal" event={"ID":"3353beadccb3550d362986f67bd77010","Type":"ContainerDied","Data":"6c01c8ff5b416646ec9344ec2389ba70503981e2fbcfd85601375c2a6289fb3d"}
Jul 09 03:43:09.729995 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.729972    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:09.729995 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.729980    2582 scope.go:117] "RemoveContainer" containerID="b45f76b5d6cde2bdbffaf966722ec58f04c9a31adbdd23fe6196cbf858789678"
Jul 09 03:43:09.730781 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.730762    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:09.730854 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.730794    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:09.730854 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.730808    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:09.731391 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:09.731042    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-164.ec2.internal\" not found" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:09.731391 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.731088    2582 scope.go:117] "RemoveContainer" containerID="6c01c8ff5b416646ec9344ec2389ba70503981e2fbcfd85601375c2a6289fb3d"
Jul 09 03:43:09.731391 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:09.731202    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-rbac-proxy-crio\" with CrashLoopBackOff: \"back-off 10s restarting failed container=kube-rbac-proxy-crio pod=kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_openshift-machine-config-operator(3353beadccb3550d362986f67bd77010)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal" podUID="3353beadccb3550d362986f67bd77010"
Jul 09 03:43:09.739516 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:09.739364    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822ebf787de3  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:BackOff,Message:Back-off restarting failed container kube-rbac-proxy-crio in pod kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_openshift-machine-config-operator(3353beadccb3550d362986f67bd77010),Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:09.731167715 +0000 UTC m=+6.550423471,LastTimestamp:2026-07-09 03:43:09.731167715 +0000 UTC m=+6.550423471,Count:1,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:09.787926 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:09.787816    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="6.4s"
Jul 09 03:43:09.990037 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.989989    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:09.991129 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.991108    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:09.991261 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.991149    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:09.991261 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.991165    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:09.991261 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:09.991201    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:10.007957 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:10.007920    2582 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:10.532097 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:10.532072    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:10.558472 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:10.558435    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 03:43:10.733230 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:10.733204    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 03:43:10.733655 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:10.733639    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:10.735780 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:10.735763    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:10.735849 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:10.735794    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:10.735849 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:10.735805    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:10.736050 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:10.736037    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-164.ec2.internal\" not found" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:10.736097 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:10.736087    2582 scope.go:117] "RemoveContainer" containerID="6c01c8ff5b416646ec9344ec2389ba70503981e2fbcfd85601375c2a6289fb3d"
Jul 09 03:43:10.736209 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:10.736194    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-rbac-proxy-crio\" with CrashLoopBackOff: \"back-off 10s restarting failed container=kube-rbac-proxy-crio pod=kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_openshift-machine-config-operator(3353beadccb3550d362986f67bd77010)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal" podUID="3353beadccb3550d362986f67bd77010"
Jul 09 03:43:10.744722 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:10.744634    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822ebf787de3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal.18c0822ebf787de3  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal,UID:3353beadccb3550d362986f67bd77010,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:BackOff,Message:Back-off restarting failed container kube-rbac-proxy-crio in pod kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_openshift-machine-config-operator(3353beadccb3550d362986f67bd77010),Source:EventSource{Component:kubelet,Host:ip-10-0-134-164.ec2.internal,},FirstTimestamp:2026-07-09 03:43:09.731167715 +0000 UTC m=+6.550423471,LastTimestamp:2026-07-09 03:43:10.736167483 +0000 UTC m=+7.555423239,Count:2,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-164.ec2.internal,}"
Jul 09 03:43:11.085531 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:11.085503    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:11.086542 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:11.086523    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:11.086652 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:11.086555    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:11.086652 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:11.086568    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:11.530514 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:11.530430    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:12.223819 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:12.223782    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 09 03:43:12.530607 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:12.530531    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:12.589963 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:12.589929    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 03:43:13.530228 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:13.530197    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:13.598068 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:13.598034    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-164.ec2.internal\" not found"
Jul 09 03:43:14.037346 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:14.037309    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 03:43:14.530458 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:14.530375    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:15.529763 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:15.529733    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:16.197444 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:16.197413    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 09 03:43:16.408171 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:16.408125    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:16.409239 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:16.409219    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:16.409344 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:16.409255    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:16.409344 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:16.409264    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:16.409344 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:16.409292    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:16.424686 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:16.424656    2582 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-134-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:16.533577 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:16.533522    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:17.530769 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:17.530734    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:18.533653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:18.533619    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 03:43:19.344659 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.344629    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bphmt"
Jul 09 03:43:19.461683 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.461635    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 03:43:19.547897 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.547856    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:19.565617 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.565593    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:19.622942 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.622887    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:19.734359 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.734333    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:19.735346 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.735329    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:19.735439 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.735364    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:19.735439 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.735378    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:19.882962 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.882858    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:19.882962 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:19.882908    2582 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:19.899281 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.899257    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 03:43:19.921591 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.921567    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:19.940539 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:19.940517    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:20.007433 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:20.007407    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:20.096339 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:20.096312    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 03:43:20.285156 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:20.285103    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:20.285156 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:20.285129    2582 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:20.350641 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:20.350606    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 03:38:19 +0000 UTC" deadline="2028-03-09 02:00:56.479512377 +0000 UTC"
Jul 09 03:43:20.350641 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:20.350635    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14614h17m36.128879999s"
Jul 09 03:43:20.540664 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:20.540590    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:20.555605 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:20.555580    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:20.613132 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:20.613105    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:20.679578 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:20.679551    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 03:43:20.892753 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:20.892669    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:20.892753 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:20.892695    2582 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:21.519937 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:21.519897    2582 apiserver.go:52] "Watching apiserver"
Jul 09 03:43:21.524369 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:21.524351    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 03:43:21.524463 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:21.524448    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=[]
Jul 09 03:43:21.531812 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:21.531790    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 03:43:22.955020 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:22.954987    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:22.971355 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:22.971330    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:23.030486 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.030457    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:23.204537 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.204502    2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-164.ec2.internal\" not found" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:23.289245 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.289217    2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:23.289245 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.289244    2582 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-134-164.ec2.internal" not found
Jul 09 03:43:23.425304 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.425261    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 03:43:23.426352 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.426336    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 03:43:23.426428 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.426365    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 03:43:23.426428 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.426375    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeHasSufficientPID"
Jul 09 03:43:23.426428 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.426402    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:23.435489 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.435461    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-164.ec2.internal"
Jul 09 03:43:23.435598 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.435492    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-164.ec2.internal\": node \"ip-10-0-134-164.ec2.internal\" not found"
Jul 09 03:43:23.455670 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.455638    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-942xr"]
Jul 09 03:43:23.457914 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.457893    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-x655t"]
Jul 09 03:43:23.458049 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.457950    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.460026 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.460006    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x655t"
Jul 09 03:43:23.460528 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.460507    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 03:43:23.460647 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.460631    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 03:43:23.460705 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.460634    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 03:43:23.460818 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.460800    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 03:43:23.461627 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.461611    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-d5nxv\""
Jul 09 03:43:23.462317 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.462301    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 03:43:23.463167 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.462998    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 03:43:23.463167 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.463007    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 03:43:23.463167 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.463112    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7vmwv\""
Jul 09 03:43:23.531240 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.531207    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:23.540792 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.540729    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 03:43:23.550010 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.549975    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 03:43:23.550147 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.550056    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-164.ec2.internal"
Jul 09 03:43:23.550269 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.550249    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal"]
Jul 09 03:43:23.550692 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.550675    2582 scope.go:117] "RemoveContainer" containerID="6c01c8ff5b416646ec9344ec2389ba70503981e2fbcfd85601375c2a6289fb3d"
Jul 09 03:43:23.555481 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.555447    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 03:43:23.557053 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.557034    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-r42mb"]
Jul 09 03:43:23.559517 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.559496    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:23.559603 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.559582    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:23.562706 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.562686    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"]
Jul 09 03:43:23.567253 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.567235    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.569686 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.569665    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 03:43:23.569686 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.569674    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g2r4j\""
Jul 09 03:43:23.569838 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.569707    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 03:43:23.570097 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.570079    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 03:43:23.582454 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.582414    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bl5th"
Jul 09 03:43:23.592562 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.592531    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bl5th"
Jul 09 03:43:23.638753 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.638719    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-socket-dir-parent\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.638753 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.638750    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-run-multus-certs\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.638960 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.638773    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-system-cni-dir\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.638960 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.638799    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-var-lib-cni-bin\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.638960 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.638822    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-var-lib-kubelet\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.638960 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.638845    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-daemon-config\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.638960 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.638914    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-299cr\" (UniqueName: \"kubernetes.io/projected/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-kube-api-access-299cr\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.638960 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.638942    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-cnibin\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.638960 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.638959    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-etc-kubernetes\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.639303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.638974    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2af28c4-c7f0-4545-9b84-04f7ec3c9533-serviceca\") pod \"node-ca-x655t\" (UID: \"c2af28c4-c7f0-4545-9b84-04f7ec3c9533\") " pod="openshift-image-registry/node-ca-x655t"
Jul 09 03:43:23.639303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.638994    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-os-release\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.639303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.639017    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-run-k8s-cni-cncf-io\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.639303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.639073    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-run-netns\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.639303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.639132    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-var-lib-cni-multus\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.639303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.639159    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-hostroot\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.639303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.639185    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-conf-dir\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.639303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.639210    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2af28c4-c7f0-4545-9b84-04f7ec3c9533-host\") pod \"node-ca-x655t\" (UID: \"c2af28c4-c7f0-4545-9b84-04f7ec3c9533\") " pod="openshift-image-registry/node-ca-x655t"
Jul 09 03:43:23.639303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.639244    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhplf\" (UniqueName: \"kubernetes.io/projected/c2af28c4-c7f0-4545-9b84-04f7ec3c9533-kube-api-access-qhplf\") pod \"node-ca-x655t\" (UID: \"c2af28c4-c7f0-4545-9b84-04f7ec3c9533\") " pod="openshift-image-registry/node-ca-x655t"
Jul 09 03:43:23.639303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.639273    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-cni-dir\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.639303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.639297    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-cni-binary-copy\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.647981 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.647956    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-164.ec2.internal"]
Jul 09 03:43:23.648194 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.648179    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 03:43:23.657564 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.657539    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-sr9tx"]
Jul 09 03:43:23.660025 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.660002    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sr9tx"
Jul 09 03:43:23.662029 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.662007    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-l2pkz"]
Jul 09 03:43:23.662859 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.662837    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 03:43:23.663001 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.662837    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 03:43:23.663001 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.662906    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dvd9f\""
Jul 09 03:43:23.663618 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.663600    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 03:43:23.664993 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.664972    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.667449 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.667431    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 03:43:23.667548 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.667433    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-drbm9\""
Jul 09 03:43:23.667742 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.667724    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 03:43:23.667742 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.667733    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 03:43:23.667971 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.667948    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 03:43:23.668081 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.667993    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 03:43:23.668081 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.668006    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 03:43:23.683504 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.683476    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-24dwk"]
Jul 09 03:43:23.686309 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.686292    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-czw4n"]
Jul 09 03:43:23.686438 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.686422    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.689022 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.689003    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9fhjl\""
Jul 09 03:43:23.689110 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.689004    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 03:43:23.689110 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.689004    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 03:43:23.689316 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.689294    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:23.689411 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.689369    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:23.693570 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.693515    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-164.ec2.internal" podStartSLOduration=0.693502538 podStartE2EDuration="693.502538ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:23.693128727 +0000 UTC m=+20.512384507" watchObservedRunningTime="2026-07-09 03:43:23.693502538 +0000 UTC m=+20.512758316"
Jul 09 03:43:23.699659 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.699638    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-4bcnc"]
Jul 09 03:43:23.702278 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.702260    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:23.704877 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.704846    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vvndz\""
Jul 09 03:43:23.704985 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.704883    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 03:43:23.705031 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.704995    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 03:43:23.739775 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739737    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-device-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.739775 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739778    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-os-release\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739799    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-run-k8s-cni-cncf-io\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-run-netns\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739832    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2af28c4-c7f0-4545-9b84-04f7ec3c9533-host\") pod \"node-ca-x655t\" (UID: \"c2af28c4-c7f0-4545-9b84-04f7ec3c9533\") " pod="openshift-image-registry/node-ca-x655t"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739851    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhplf\" (UniqueName: \"kubernetes.io/projected/c2af28c4-c7f0-4545-9b84-04f7ec3c9533-kube-api-access-qhplf\") pod \"node-ca-x655t\" (UID: \"c2af28c4-c7f0-4545-9b84-04f7ec3c9533\") " pod="openshift-image-registry/node-ca-x655t"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739892    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-sys-fs\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739900    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-run-netns\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739911    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-cni-binary-copy\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739900    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-run-k8s-cni-cncf-io\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739931    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-run-multus-certs\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739904    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-os-release\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739940    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2af28c4-c7f0-4545-9b84-04f7ec3c9533-host\") pod \"node-ca-x655t\" (UID: \"c2af28c4-c7f0-4545-9b84-04f7ec3c9533\") " pod="openshift-image-registry/node-ca-x655t"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739953    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739969    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-registration-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739979    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-run-multus-certs\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.739987    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-var-lib-kubelet\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740004    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-daemon-config\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740039    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-299cr\" (UniqueName: \"kubernetes.io/projected/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-kube-api-access-299cr\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740073    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-cnibin\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740100    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2af28c4-c7f0-4545-9b84-04f7ec3c9533-serviceca\") pod \"node-ca-x655t\" (UID: \"c2af28c4-c7f0-4545-9b84-04f7ec3c9533\") " pod="openshift-image-registry/node-ca-x655t"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740150    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-var-lib-kubelet\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-var-lib-cni-multus\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740257    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-hostroot\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740286    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-conf-dir\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740280    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-var-lib-cni-multus\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740348    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-conf-dir\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740343    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-hostroot\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740359    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-cnibin\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-socket-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740399    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-cni-dir\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740431    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-socket-dir-parent\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740473    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-system-cni-dir\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740480    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-socket-dir-parent\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740503    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-cni-binary-copy\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.740735 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740516    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-var-lib-cni-bin\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.741287 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740520    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-system-cni-dir\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.741287 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740476    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-cni-dir\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.741287 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740547    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47jkl\" (UniqueName: \"kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl\") pod \"network-check-target-r42mb\" (UID: \"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc\") " pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:23.741287 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740544    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-host-var-lib-cni-bin\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.741287 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740580    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8sms\" (UniqueName: \"kubernetes.io/projected/ba4762bd-3392-4767-9a9d-c377797a8aaa-kube-api-access-h8sms\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.741287 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740604    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-etc-kubernetes\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.741287 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.740647    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-etc-kubernetes\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.741497 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.741486    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-multus-daemon-config\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.741708 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.741692    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2af28c4-c7f0-4545-9b84-04f7ec3c9533-serviceca\") pod \"node-ca-x655t\" (UID: \"c2af28c4-c7f0-4545-9b84-04f7ec3c9533\") " pod="openshift-image-registry/node-ca-x655t"
Jul 09 03:43:23.748451 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.748427    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 03:43:23.752906 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.752884    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-299cr\" (UniqueName: \"kubernetes.io/projected/ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72-kube-api-access-299cr\") pod \"multus-942xr\" (UID: \"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72\") " pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.753973 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.753948    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhplf\" (UniqueName: \"kubernetes.io/projected/c2af28c4-c7f0-4545-9b84-04f7ec3c9533-kube-api-access-qhplf\") pod \"node-ca-x655t\" (UID: \"c2af28c4-c7f0-4545-9b84-04f7ec3c9533\") " pod="openshift-image-registry/node-ca-x655t"
Jul 09 03:43:23.754804 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.754782    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-2wsms"]
Jul 09 03:43:23.755375 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.755356    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 03:43:23.758465 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.758438    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal" event={"ID":"3353beadccb3550d362986f67bd77010","Type":"ContainerStarted","Data":"39cbdef1b211de68bb34650c0d5cc511a0282119026ace4bc9172300851a4a62"}
Jul 09 03:43:23.758580 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.758540    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2wsms"
Jul 09 03:43:23.759087 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.759070    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-fjm27"]
Jul 09 03:43:23.760963 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.760943    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2htfn\""
Jul 09 03:43:23.761052 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.760985    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 03:43:23.761052 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.760992    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 03:43:23.761480 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.761467    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.763778 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.763762    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 03:43:23.763883 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.763852    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 03:43:23.763953 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.763915    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cf7vs\""
Jul 09 03:43:23.768625 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.768608    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-942xr"
Jul 09 03:43:23.773328 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.773308    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x655t"
Jul 09 03:43:23.774027 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.774006    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-pw8dq"]
Jul 09 03:43:23.776583 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.776566    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:23.776675 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.776637    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:23.777404 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:23.777385    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8e2cd2_4854_4d43_be1a_9a0c1bf3ce72.slice/crio-4e9092411ceda8441f936faa03a2ceec2554f09d07d7ee6d9c42f5095d67971c WatchSource:0}: Error finding container 4e9092411ceda8441f936faa03a2ceec2554f09d07d7ee6d9c42f5095d67971c: Status 404 returned error can't find the container with id 4e9092411ceda8441f936faa03a2ceec2554f09d07d7ee6d9c42f5095d67971c
Jul 09 03:43:23.783121 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:23.783096    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2af28c4_c7f0_4545_9b84_04f7ec3c9533.slice/crio-600a0fd8aaa4e6a51fd05462a698b5b21108b4d677ce5e24c7fd120b44ec1778 WatchSource:0}: Error finding container 600a0fd8aaa4e6a51fd05462a698b5b21108b4d677ce5e24c7fd120b44ec1778: Status 404 returned error can't find the container with id 600a0fd8aaa4e6a51fd05462a698b5b21108b4d677ce5e24c7fd120b44ec1778
Jul 09 03:43:23.840399 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.840279    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal" podStartSLOduration=0.840263055 podStartE2EDuration="840.263055ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:23.840219756 +0000 UTC m=+20.659475533" watchObservedRunningTime="2026-07-09 03:43:23.840263055 +0000 UTC m=+20.659518833"
Jul 09 03:43:23.841172 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.841265 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841190    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-os-release\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.841265 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841217    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-systemd-units\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841265 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-slash\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841265 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841260    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98de9f69-0044-4adc-879b-5eb059d178ad-ovnkube-config\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841435 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841286    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjh2q\" (UniqueName: \"kubernetes.io/projected/15c1454e-931d-4ab2-8bcc-20db5808084e-kube-api-access-pjh2q\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:23.841435 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841323    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-socket-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.841435 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841333    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.841435 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841344    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-etc-openvswitch\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841435 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841360    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98de9f69-0044-4adc-879b-5eb059d178ad-env-overrides\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841435 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-run-systemd\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841435 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-var-lib-openvswitch\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841435 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841408    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f-agent-certs\") pod \"konnectivity-agent-4bcnc\" (UID: \"2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f\") " pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:23.841435 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841425    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8sms\" (UniqueName: \"kubernetes.io/projected/ba4762bd-3392-4767-9a9d-c377797a8aaa-kube-api-access-h8sms\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.841758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841473    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-socket-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.841758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841543    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-system-cni-dir\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.841758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841560    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be397f8d-fd42-44ea-b444-3530fe24606b-cni-binary-copy\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.841758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841576    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-run-ovn\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841591    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98de9f69-0044-4adc-879b-5eb059d178ad-ovn-node-metrics-cert\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841626    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-run-openvswitch\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841682    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-device-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.841758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841711    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.841758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841737    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-node-log\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.841758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841752    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-device-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841759    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-cni-netd\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841790    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75bfd683-da7d-4dfc-8179-884e0455367e-host-slash\") pod \"iptables-alerter-sr9tx\" (UID: \"75bfd683-da7d-4dfc-8179-884e0455367e\") " pod="openshift-network-operator/iptables-alerter-sr9tx"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841814    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-cnibin\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841840    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/be397f8d-fd42-44ea-b444-3530fe24606b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn5gd\" (UniqueName: \"kubernetes.io/projected/75bfd683-da7d-4dfc-8179-884e0455367e-kube-api-access-qn5gd\") pod \"iptables-alerter-sr9tx\" (UID: \"75bfd683-da7d-4dfc-8179-884e0455367e\") " pod="openshift-network-operator/iptables-alerter-sr9tx"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841921    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841947    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt6r8\" (UniqueName: \"kubernetes.io/projected/be397f8d-fd42-44ea-b444-3530fe24606b-kube-api-access-dt6r8\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841970    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.841997    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f-konnectivity-ca\") pod \"konnectivity-agent-4bcnc\" (UID: \"2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f\") " pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842023    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47jkl\" (UniqueName: \"kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl\") pod \"network-check-target-r42mb\" (UID: \"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc\") " pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842039    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-log-socket\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842060    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-cni-bin\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842077    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/75bfd683-da7d-4dfc-8179-884e0455367e-iptables-alerter-script\") pod \"iptables-alerter-sr9tx\" (UID: \"75bfd683-da7d-4dfc-8179-884e0455367e\") " pod="openshift-network-operator/iptables-alerter-sr9tx"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842096    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/be397f8d-fd42-44ea-b444-3530fe24606b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842111    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-kubelet\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.842277 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842133    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-sys-fs\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.842950 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842153    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.842950 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842167    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-registration-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.842950 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842183    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9srl\" (UniqueName: \"kubernetes.io/projected/98de9f69-0044-4adc-879b-5eb059d178ad-kube-api-access-g9srl\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.842950 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842193    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.842950 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842197    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-run-netns\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.842950 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842218    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98de9f69-0044-4adc-879b-5eb059d178ad-ovnkube-script-lib\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.842950 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842227    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-registration-dir\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.842950 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.842277    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ba4762bd-3392-4767-9a9d-c377797a8aaa-sys-fs\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.848603 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.848583    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 03:43:23.848684 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.848612    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 03:43:23.848684 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.848625    2582 projected.go:196] Error preparing data for projected volume kube-api-access-47jkl for pod openshift-network-diagnostics/network-check-target-r42mb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:23.848745 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.848700    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl podName:3aa6f0a7-69fd-47a6-92c4-3bb5132afddc nodeName:}" failed. No retries permitted until 2026-07-09 03:43:24.348671538 +0000 UTC m=+21.167927306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-47jkl" (UniqueName: "kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl") pod "network-check-target-r42mb" (UID: "3aa6f0a7-69fd-47a6-92c4-3bb5132afddc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:23.850469 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.850452    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8sms\" (UniqueName: \"kubernetes.io/projected/ba4762bd-3392-4767-9a9d-c377797a8aaa-kube-api-access-h8sms\") pod \"aws-ebs-csi-driver-node-zjntg\" (UID: \"ba4762bd-3392-4767-9a9d-c377797a8aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.876436 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.876392    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg"
Jul 09 03:43:23.886258 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:23.886230    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4762bd_3392_4767_9a9d_c377797a8aaa.slice/crio-c56fc282eb2efd9d1e5270848c8b873db60b1ed69beeaf62c864888e8fae9af7 WatchSource:0}: Error finding container c56fc282eb2efd9d1e5270848c8b873db60b1ed69beeaf62c864888e8fae9af7: Status 404 returned error can't find the container with id c56fc282eb2efd9d1e5270848c8b873db60b1ed69beeaf62c864888e8fae9af7
Jul 09 03:43:23.943242 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943208    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-node-log\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.943335 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943247    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-cni-netd\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.943335 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943268    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75bfd683-da7d-4dfc-8179-884e0455367e-host-slash\") pod \"iptables-alerter-sr9tx\" (UID: \"75bfd683-da7d-4dfc-8179-884e0455367e\") " pod="openshift-network-operator/iptables-alerter-sr9tx"
Jul 09 03:43:23.943335 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943313    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75bfd683-da7d-4dfc-8179-884e0455367e-host-slash\") pod \"iptables-alerter-sr9tx\" (UID: \"75bfd683-da7d-4dfc-8179-884e0455367e\") " pod="openshift-network-operator/iptables-alerter-sr9tx"
Jul 09 03:43:23.943335 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943319    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-node-log\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.943474 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943343    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-cnibin\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.943474 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943370    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/be397f8d-fd42-44ea-b444-3530fe24606b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.943474 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943391    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-cnibin\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.943474 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943319    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-cni-netd\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.943474 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943398    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn5gd\" (UniqueName: \"kubernetes.io/projected/75bfd683-da7d-4dfc-8179-884e0455367e-kube-api-access-qn5gd\") pod \"iptables-alerter-sr9tx\" (UID: \"75bfd683-da7d-4dfc-8179-884e0455367e\") " pod="openshift-network-operator/iptables-alerter-sr9tx"
Jul 09 03:43:23.943474 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943433    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.943474 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943455    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt6r8\" (UniqueName: \"kubernetes.io/projected/be397f8d-fd42-44ea-b444-3530fe24606b-kube-api-access-dt6r8\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.943698 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943486    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-sysctl-conf\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.943698 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943511    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.943698 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943603    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:23.943698 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943639    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f-konnectivity-ca\") pod \"konnectivity-agent-4bcnc\" (UID: \"2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f\") " pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:23.943698 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943664    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-sysctl-d\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-lib-modules\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.943729    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-log-socket\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943777    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-cni-bin\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:23.943801    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs podName:15c1454e-931d-4ab2-8bcc-20db5808084e nodeName:}" failed. No retries permitted until 2026-07-09 03:43:24.443783832 +0000 UTC m=+21.263039607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs") pod "network-metrics-daemon-czw4n" (UID: "15c1454e-931d-4ab2-8bcc-20db5808084e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943819    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-cni-bin\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943815    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-log-socket\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943831    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/75bfd683-da7d-4dfc-8179-884e0455367e-iptables-alerter-script\") pod \"iptables-alerter-sr9tx\" (UID: \"75bfd683-da7d-4dfc-8179-884e0455367e\") " pod="openshift-network-operator/iptables-alerter-sr9tx"
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/be397f8d-fd42-44ea-b444-3530fe24606b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943937    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cnfh\" (UniqueName: \"kubernetes.io/projected/fd5a5f67-7cb7-4196-b7ef-8f3227eed070-kube-api-access-2cnfh\") pod \"node-resolver-2wsms\" (UID: \"fd5a5f67-7cb7-4196-b7ef-8f3227eed070\") " pod="openshift-dns/node-resolver-2wsms"
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943962    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56157167-4b57-4f59-9f0b-8925fea4b4b3-dbus\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.943989    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-tuned\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.944011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944013    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-kubelet\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944018    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/be397f8d-fd42-44ea-b444-3530fe24606b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944057    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-kubelet\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944070    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9srl\" (UniqueName: \"kubernetes.io/projected/98de9f69-0044-4adc-879b-5eb059d178ad-kube-api-access-g9srl\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd5a5f67-7cb7-4196-b7ef-8f3227eed070-tmp-dir\") pod \"node-resolver-2wsms\" (UID: \"fd5a5f67-7cb7-4196-b7ef-8f3227eed070\") " pod="openshift-dns/node-resolver-2wsms"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944129    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-systemd\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944148    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-run-netns\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944165    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98de9f69-0044-4adc-879b-5eb059d178ad-ovnkube-script-lib\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944191    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944214    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-run\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944222    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-run-netns\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944233    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f-konnectivity-ca\") pod \"konnectivity-agent-4bcnc\" (UID: \"2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f\") " pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944278    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mjl7\" (UniqueName: \"kubernetes.io/projected/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-kube-api-access-4mjl7\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944320    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-os-release\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944351    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-systemd-units\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944375    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-slash\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944398    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-systemd-units\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.944581 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944401    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98de9f69-0044-4adc-879b-5eb059d178ad-ovnkube-config\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944408    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-os-release\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944408    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-slash\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944446    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/75bfd683-da7d-4dfc-8179-884e0455367e-iptables-alerter-script\") pod \"iptables-alerter-sr9tx\" (UID: \"75bfd683-da7d-4dfc-8179-884e0455367e\") " pod="openshift-network-operator/iptables-alerter-sr9tx"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjh2q\" (UniqueName: \"kubernetes.io/projected/15c1454e-931d-4ab2-8bcc-20db5808084e-kube-api-access-pjh2q\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944523    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fd5a5f67-7cb7-4196-b7ef-8f3227eed070-hosts-file\") pod \"node-resolver-2wsms\" (UID: \"fd5a5f67-7cb7-4196-b7ef-8f3227eed070\") " pod="openshift-dns/node-resolver-2wsms"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944551    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-etc-openvswitch\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98de9f69-0044-4adc-879b-5eb059d178ad-env-overrides\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944616    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-modprobe-d\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944663    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-etc-openvswitch\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944676    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-tmp\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944755    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-run-systemd\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944756    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98de9f69-0044-4adc-879b-5eb059d178ad-ovnkube-script-lib\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944780    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-var-lib-openvswitch\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944803    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f-agent-certs\") pod \"konnectivity-agent-4bcnc\" (UID: \"2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f\") " pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944814    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-run-systemd\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944825    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-var-lib-kubelet\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.945248 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944842    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-var-lib-openvswitch\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944855    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944910    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56157167-4b57-4f59-9f0b-8925fea4b4b3-kubelet-config\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944934    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-kubernetes\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944974    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98de9f69-0044-4adc-879b-5eb059d178ad-ovnkube-config\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944973    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944994    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-sys\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.944995    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98de9f69-0044-4adc-879b-5eb059d178ad-env-overrides\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945026    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-host\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945046    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/be397f8d-fd42-44ea-b444-3530fe24606b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945053    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-system-cni-dir\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945091    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-system-cni-dir\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945117    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be397f8d-fd42-44ea-b444-3530fe24606b-cni-binary-copy\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945141    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-run-ovn\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945156    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98de9f69-0044-4adc-879b-5eb059d178ad-ovn-node-metrics-cert\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945186    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-run-openvswitch\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945232    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.946059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945260    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-run-openvswitch\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.946922 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-sysconfig\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:23.946922 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945280    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98de9f69-0044-4adc-879b-5eb059d178ad-run-ovn\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.946922 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945404    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be397f8d-fd42-44ea-b444-3530fe24606b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.946922 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.945523    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be397f8d-fd42-44ea-b444-3530fe24606b-cni-binary-copy\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.947244 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.947224    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98de9f69-0044-4adc-879b-5eb059d178ad-ovn-node-metrics-cert\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.947484 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.947466    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f-agent-certs\") pod \"konnectivity-agent-4bcnc\" (UID: \"2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f\") " pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:23.952535 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.952511    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt6r8\" (UniqueName: \"kubernetes.io/projected/be397f8d-fd42-44ea-b444-3530fe24606b-kube-api-access-dt6r8\") pod \"multus-additional-cni-plugins-24dwk\" (UID: \"be397f8d-fd42-44ea-b444-3530fe24606b\") " pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:23.952620 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.952594    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn5gd\" (UniqueName: \"kubernetes.io/projected/75bfd683-da7d-4dfc-8179-884e0455367e-kube-api-access-qn5gd\") pod \"iptables-alerter-sr9tx\" (UID: \"75bfd683-da7d-4dfc-8179-884e0455367e\") " pod="openshift-network-operator/iptables-alerter-sr9tx"
Jul 09 03:43:23.953166 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.953150    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9srl\" (UniqueName: \"kubernetes.io/projected/98de9f69-0044-4adc-879b-5eb059d178ad-kube-api-access-g9srl\") pod \"ovnkube-node-l2pkz\" (UID: \"98de9f69-0044-4adc-879b-5eb059d178ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.956784 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.956760    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjh2q\" (UniqueName: \"kubernetes.io/projected/15c1454e-931d-4ab2-8bcc-20db5808084e-kube-api-access-pjh2q\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:23.970878 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.970840    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sr9tx"
Jul 09 03:43:23.975043 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.974926    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:23.982853 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:23.982829    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75bfd683_da7d_4dfc_8179_884e0455367e.slice/crio-5295b5646a83fc7f65a079a3998beb0969ed07efef445d2f302812524f758970 WatchSource:0}: Error finding container 5295b5646a83fc7f65a079a3998beb0969ed07efef445d2f302812524f758970: Status 404 returned error can't find the container with id 5295b5646a83fc7f65a079a3998beb0969ed07efef445d2f302812524f758970
Jul 09 03:43:23.984801 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:23.984774    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98de9f69_0044_4adc_879b_5eb059d178ad.slice/crio-c7a48a50f4f491828ff28f111c4209785292c5f3bde15815776b70a198345f87 WatchSource:0}: Error finding container c7a48a50f4f491828ff28f111c4209785292c5f3bde15815776b70a198345f87: Status 404 returned error can't find the container with id c7a48a50f4f491828ff28f111c4209785292c5f3bde15815776b70a198345f87
Jul 09 03:43:23.993851 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:23.993829    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-24dwk"
Jul 09 03:43:24.004398 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:24.004374    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe397f8d_fd42_44ea_b444_3530fe24606b.slice/crio-a94137cf607bbf9a9243d9f466aefade9e84390307159a9f96e58dab0c52e60a WatchSource:0}: Error finding container a94137cf607bbf9a9243d9f466aefade9e84390307159a9f96e58dab0c52e60a: Status 404 returned error can't find the container with id a94137cf607bbf9a9243d9f466aefade9e84390307159a9f96e58dab0c52e60a
Jul 09 03:43:24.009437 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.009417    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:24.017590 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:24.017568    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a8cd1aa_e1ab_43b7_b1d5_0c91cd7b805f.slice/crio-19a35355c3b6a82b8f91d6c1cbac1a9c25332cb287be3ae88c66222b150f806d WatchSource:0}: Error finding container 19a35355c3b6a82b8f91d6c1cbac1a9c25332cb287be3ae88c66222b150f806d: Status 404 returned error can't find the container with id 19a35355c3b6a82b8f91d6c1cbac1a9c25332cb287be3ae88c66222b150f806d
Jul 09 03:43:24.046185 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046149    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fd5a5f67-7cb7-4196-b7ef-8f3227eed070-hosts-file\") pod \"node-resolver-2wsms\" (UID: \"fd5a5f67-7cb7-4196-b7ef-8f3227eed070\") " pod="openshift-dns/node-resolver-2wsms"
Jul 09 03:43:24.046307 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-modprobe-d\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046307 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046220    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-tmp\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046307 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046268    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fd5a5f67-7cb7-4196-b7ef-8f3227eed070-hosts-file\") pod \"node-resolver-2wsms\" (UID: \"fd5a5f67-7cb7-4196-b7ef-8f3227eed070\") " pod="openshift-dns/node-resolver-2wsms"
Jul 09 03:43:24.046423 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046326    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-var-lib-kubelet\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046423 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046341    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-modprobe-d\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046423 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046366    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56157167-4b57-4f59-9f0b-8925fea4b4b3-kubelet-config\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:24.046423 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046378    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-var-lib-kubelet\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046423 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046386    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-kubernetes\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046423 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046416    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-sys\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046423 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046424    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-kubernetes\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046442    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-host\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046463    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-sys\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046481    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-sysconfig\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046487    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56157167-4b57-4f59-9f0b-8925fea4b4b3-kubelet-config\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046503    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-host\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046506    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-sysctl-conf\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046564    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-sysctl-d\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046584    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-sysconfig\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046589    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-lib-modules\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046588    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-sysctl-conf\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046647    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cnfh\" (UniqueName: \"kubernetes.io/projected/fd5a5f67-7cb7-4196-b7ef-8f3227eed070-kube-api-access-2cnfh\") pod \"node-resolver-2wsms\" (UID: \"fd5a5f67-7cb7-4196-b7ef-8f3227eed070\") " pod="openshift-dns/node-resolver-2wsms"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046670    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56157167-4b57-4f59-9f0b-8925fea4b4b3-dbus\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046669    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-sysctl-d\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046688    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-tuned\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046719    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd5a5f67-7cb7-4196-b7ef-8f3227eed070-tmp-dir\") pod \"node-resolver-2wsms\" (UID: \"fd5a5f67-7cb7-4196-b7ef-8f3227eed070\") " pod="openshift-dns/node-resolver-2wsms"
Jul 09 03:43:24.046738 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-systemd\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.047387 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046761    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:24.047387 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046767    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-lib-modules\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.047387 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046772    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56157167-4b57-4f59-9f0b-8925fea4b4b3-dbus\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:24.047387 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046784    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-run\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.047387 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046810    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mjl7\" (UniqueName: \"kubernetes.io/projected/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-kube-api-access-4mjl7\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.047387 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046825    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-systemd\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.047387 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:24.046859    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:24.047387 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.046911    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-run\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.047387 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:24.046937    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret podName:56157167-4b57-4f59-9f0b-8925fea4b4b3 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:24.546915318 +0000 UTC m=+21.366171093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret") pod "global-pull-secret-syncer-pw8dq" (UID: "56157167-4b57-4f59-9f0b-8925fea4b4b3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:24.047387 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.047147    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd5a5f67-7cb7-4196-b7ef-8f3227eed070-tmp-dir\") pod \"node-resolver-2wsms\" (UID: \"fd5a5f67-7cb7-4196-b7ef-8f3227eed070\") " pod="openshift-dns/node-resolver-2wsms"
Jul 09 03:43:24.048974 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.048955    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-tmp\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.049085 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.048998    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-etc-tuned\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.057141 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.057110    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cnfh\" (UniqueName: \"kubernetes.io/projected/fd5a5f67-7cb7-4196-b7ef-8f3227eed070-kube-api-access-2cnfh\") pod \"node-resolver-2wsms\" (UID: \"fd5a5f67-7cb7-4196-b7ef-8f3227eed070\") " pod="openshift-dns/node-resolver-2wsms"
Jul 09 03:43:24.057600 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.057584    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mjl7\" (UniqueName: \"kubernetes.io/projected/9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389-kube-api-access-4mjl7\") pod \"tuned-fjm27\" (UID: \"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389\") " pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.067395 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.067372    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2wsms"
Jul 09 03:43:24.072363 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.072339    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fjm27"
Jul 09 03:43:24.077295 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:24.077272    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd5a5f67_7cb7_4196_b7ef_8f3227eed070.slice/crio-dbef924ec00c30ef0866f2e1390882efa5a7e04fa3881e13e72f0f55a3da69ca WatchSource:0}: Error finding container dbef924ec00c30ef0866f2e1390882efa5a7e04fa3881e13e72f0f55a3da69ca: Status 404 returned error can't find the container with id dbef924ec00c30ef0866f2e1390882efa5a7e04fa3881e13e72f0f55a3da69ca
Jul 09 03:43:24.081276 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:24.081254    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8b0c0d_fb9f_4e43_8cda_ffeac9c9b389.slice/crio-6f32f91b0b1afd106dff965a9ac8fd405d515015d24eadb4bb8551e48dca12c7 WatchSource:0}: Error finding container 6f32f91b0b1afd106dff965a9ac8fd405d515015d24eadb4bb8551e48dca12c7: Status 404 returned error can't find the container with id 6f32f91b0b1afd106dff965a9ac8fd405d515015d24eadb4bb8551e48dca12c7
Jul 09 03:43:24.449282 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.448522    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:24.449282 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.448578    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47jkl\" (UniqueName: \"kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl\") pod \"network-check-target-r42mb\" (UID: \"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc\") " pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:24.449282 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:24.448779    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 03:43:24.449282 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:24.448798    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 03:43:24.449282 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:24.448815    2582 projected.go:196] Error preparing data for projected volume kube-api-access-47jkl for pod openshift-network-diagnostics/network-check-target-r42mb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:24.449282 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:24.448892    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl podName:3aa6f0a7-69fd-47a6-92c4-3bb5132afddc nodeName:}" failed. No retries permitted until 2026-07-09 03:43:25.448858165 +0000 UTC m=+22.268113935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-47jkl" (UniqueName: "kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl") pod "network-check-target-r42mb" (UID: "3aa6f0a7-69fd-47a6-92c4-3bb5132afddc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:24.449838 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:24.449358    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:24.449838 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:24.449438    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs podName:15c1454e-931d-4ab2-8bcc-20db5808084e nodeName:}" failed. No retries permitted until 2026-07-09 03:43:25.449418192 +0000 UTC m=+22.268673952 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs") pod "network-metrics-daemon-czw4n" (UID: "15c1454e-931d-4ab2-8bcc-20db5808084e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:24.549893 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.549835    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:24.550089 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:24.549992    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:24.550089 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:24.550049    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret podName:56157167-4b57-4f59-9f0b-8925fea4b4b3 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:25.550034413 +0000 UTC m=+22.369290170 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret") pod "global-pull-secret-syncer-pw8dq" (UID: "56157167-4b57-4f59-9f0b-8925fea4b4b3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:24.596106 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.595735    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 03:38:23 +0000 UTC" deadline="2028-02-11 22:58:45.433101101 +0000 UTC"
Jul 09 03:43:24.596106 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.596036    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13987h15m20.837072758s"
Jul 09 03:43:24.765657 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.765566    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fjm27" event={"ID":"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389","Type":"ContainerStarted","Data":"6f32f91b0b1afd106dff965a9ac8fd405d515015d24eadb4bb8551e48dca12c7"}
Jul 09 03:43:24.769550 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.769460    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4bcnc" event={"ID":"2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f","Type":"ContainerStarted","Data":"19a35355c3b6a82b8f91d6c1cbac1a9c25332cb287be3ae88c66222b150f806d"}
Jul 09 03:43:24.774583 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.774495    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24dwk" event={"ID":"be397f8d-fd42-44ea-b444-3530fe24606b","Type":"ContainerStarted","Data":"a94137cf607bbf9a9243d9f466aefade9e84390307159a9f96e58dab0c52e60a"}
Jul 09 03:43:24.778597 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.778458    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz" event={"ID":"98de9f69-0044-4adc-879b-5eb059d178ad","Type":"ContainerStarted","Data":"c7a48a50f4f491828ff28f111c4209785292c5f3bde15815776b70a198345f87"}
Jul 09 03:43:24.781237 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.781202    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sr9tx" event={"ID":"75bfd683-da7d-4dfc-8179-884e0455367e","Type":"ContainerStarted","Data":"5295b5646a83fc7f65a079a3998beb0969ed07efef445d2f302812524f758970"}
Jul 09 03:43:24.784250 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.784216    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2wsms" event={"ID":"fd5a5f67-7cb7-4196-b7ef-8f3227eed070","Type":"ContainerStarted","Data":"dbef924ec00c30ef0866f2e1390882efa5a7e04fa3881e13e72f0f55a3da69ca"}
Jul 09 03:43:24.787787 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.787755    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg" event={"ID":"ba4762bd-3392-4767-9a9d-c377797a8aaa","Type":"ContainerStarted","Data":"c56fc282eb2efd9d1e5270848c8b873db60b1ed69beeaf62c864888e8fae9af7"}
Jul 09 03:43:24.791325 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.791295    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x655t" event={"ID":"c2af28c4-c7f0-4545-9b84-04f7ec3c9533","Type":"ContainerStarted","Data":"600a0fd8aaa4e6a51fd05462a698b5b21108b4d677ce5e24c7fd120b44ec1778"}
Jul 09 03:43:24.794555 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.794498    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-942xr" event={"ID":"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72","Type":"ContainerStarted","Data":"4e9092411ceda8441f936faa03a2ceec2554f09d07d7ee6d9c42f5095d67971c"}
Jul 09 03:43:24.828177 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:24.828142    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 03:43:25.456757 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:25.456412    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:25.456757 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:25.456463    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47jkl\" (UniqueName: \"kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl\") pod \"network-check-target-r42mb\" (UID: \"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc\") " pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:25.456757 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:25.456623    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 03:43:25.456757 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:25.456640    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 03:43:25.456757 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:25.456656    2582 projected.go:196] Error preparing data for projected volume kube-api-access-47jkl for pod openshift-network-diagnostics/network-check-target-r42mb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:25.456757 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:25.456672    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:25.456757 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:25.456719    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl podName:3aa6f0a7-69fd-47a6-92c4-3bb5132afddc nodeName:}" failed. No retries permitted until 2026-07-09 03:43:27.456702052 +0000 UTC m=+24.275957810 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-47jkl" (UniqueName: "kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl") pod "network-check-target-r42mb" (UID: "3aa6f0a7-69fd-47a6-92c4-3bb5132afddc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:25.456757 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:25.456737    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs podName:15c1454e-931d-4ab2-8bcc-20db5808084e nodeName:}" failed. No retries permitted until 2026-07-09 03:43:27.456728621 +0000 UTC m=+24.275984382 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs") pod "network-metrics-daemon-czw4n" (UID: "15c1454e-931d-4ab2-8bcc-20db5808084e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:25.557689 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:25.557644    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:25.557905 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:25.557831    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:25.557970 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:25.557910    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret podName:56157167-4b57-4f59-9f0b-8925fea4b4b3 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:27.557890697 +0000 UTC m=+24.377146459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret") pod "global-pull-secret-syncer-pw8dq" (UID: "56157167-4b57-4f59-9f0b-8925fea4b4b3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:25.596674 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:25.596630    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 03:38:23 +0000 UTC" deadline="2028-04-11 04:12:53.45575819 +0000 UTC"
Jul 09 03:43:25.596674 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:25.596674    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15408h29m27.85908939s"
Jul 09 03:43:25.709194 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:25.708592    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:25.709194 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:25.708735    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:25.709194 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:25.708882    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:25.709194 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:25.709005    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:25.709194 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:25.708592    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:25.709194 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:25.709105    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:27.475473 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:27.474784    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:27.475473 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:27.474844    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47jkl\" (UniqueName: \"kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl\") pod \"network-check-target-r42mb\" (UID: \"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc\") " pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:27.475473 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:27.475076    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 03:43:27.475473 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:27.475099    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 03:43:27.475473 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:27.475118    2582 projected.go:196] Error preparing data for projected volume kube-api-access-47jkl for pod openshift-network-diagnostics/network-check-target-r42mb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:27.475473 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:27.475181    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl podName:3aa6f0a7-69fd-47a6-92c4-3bb5132afddc nodeName:}" failed. No retries permitted until 2026-07-09 03:43:31.475161129 +0000 UTC m=+28.294416892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-47jkl" (UniqueName: "kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl") pod "network-check-target-r42mb" (UID: "3aa6f0a7-69fd-47a6-92c4-3bb5132afddc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:27.475473 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:27.475292    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:27.475473 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:27.475357    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs podName:15c1454e-931d-4ab2-8bcc-20db5808084e nodeName:}" failed. No retries permitted until 2026-07-09 03:43:31.475339042 +0000 UTC m=+28.294594812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs") pod "network-metrics-daemon-czw4n" (UID: "15c1454e-931d-4ab2-8bcc-20db5808084e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:27.575478 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:27.575430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:27.575674 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:27.575609    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:27.575750 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:27.575678    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret podName:56157167-4b57-4f59-9f0b-8925fea4b4b3 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:31.575658844 +0000 UTC m=+28.394914607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret") pod "global-pull-secret-syncer-pw8dq" (UID: "56157167-4b57-4f59-9f0b-8925fea4b4b3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:27.709716 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:27.709679    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:27.709917 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:27.709809    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:27.709917 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:27.709889    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:27.710047 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:27.709952    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:27.710047 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:27.709997    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:27.710047 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:27.710041    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:29.708413 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:29.708360    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:29.708910 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:29.708509    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:29.708910 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:29.708572    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:29.708910 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:29.708706    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:29.708910 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:29.708777    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:29.708910 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:29.708893    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:31.507950 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:31.507912    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:31.508458 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:31.507966    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47jkl\" (UniqueName: \"kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl\") pod \"network-check-target-r42mb\" (UID: \"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc\") " pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:31.508458 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:31.508133    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:31.508458 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:31.508216    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs podName:15c1454e-931d-4ab2-8bcc-20db5808084e nodeName:}" failed. No retries permitted until 2026-07-09 03:43:39.508193148 +0000 UTC m=+36.327448908 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs") pod "network-metrics-daemon-czw4n" (UID: "15c1454e-931d-4ab2-8bcc-20db5808084e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:31.508458 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:31.508141    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 03:43:31.508458 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:31.508256    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 03:43:31.508458 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:31.508270    2582 projected.go:196] Error preparing data for projected volume kube-api-access-47jkl for pod openshift-network-diagnostics/network-check-target-r42mb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:31.508458 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:31.508328    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl podName:3aa6f0a7-69fd-47a6-92c4-3bb5132afddc nodeName:}" failed. No retries permitted until 2026-07-09 03:43:39.508303127 +0000 UTC m=+36.327558902 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-47jkl" (UniqueName: "kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl") pod "network-check-target-r42mb" (UID: "3aa6f0a7-69fd-47a6-92c4-3bb5132afddc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:31.608912 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:31.608874    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:31.609109 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:31.609023    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:31.609109 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:31.609106    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret podName:56157167-4b57-4f59-9f0b-8925fea4b4b3 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:39.609085475 +0000 UTC m=+36.428341238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret") pod "global-pull-secret-syncer-pw8dq" (UID: "56157167-4b57-4f59-9f0b-8925fea4b4b3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:31.707941 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:31.707909    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:31.708119 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:31.708094    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:31.708742 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:31.708548    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:31.708742 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:31.708680    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:31.708960 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:31.708835    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:31.709022 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:31.708970    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:33.712699 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:33.711938    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:33.712699 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:33.712063    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:33.712699 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:33.712435    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:33.712699 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:33.712521    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:33.712699 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:33.712577    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:33.712699 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:33.712655    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:35.708220 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:35.708132    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:35.708220 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:35.708165    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:35.708751 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:35.708287    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:35.708751 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:35.708311    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:35.708751 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:35.708391    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:35.708751 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:35.708492    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:37.708840 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:37.708719    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:37.708840 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:37.708727    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:37.709419 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:37.708852    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:37.709419 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:37.708941    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:37.709419 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:37.709290    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:37.709419 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:37.709373    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:39.573635 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:39.573588    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:39.574184 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:39.573647    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47jkl\" (UniqueName: \"kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl\") pod \"network-check-target-r42mb\" (UID: \"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc\") " pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:39.574184 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:39.573772    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:39.574184 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:39.573794    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 03:43:39.574184 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:39.573813    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 03:43:39.574184 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:39.573826    2582 projected.go:196] Error preparing data for projected volume kube-api-access-47jkl for pod openshift-network-diagnostics/network-check-target-r42mb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:39.574184 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:39.573897    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl podName:3aa6f0a7-69fd-47a6-92c4-3bb5132afddc nodeName:}" failed. No retries permitted until 2026-07-09 03:43:55.573876343 +0000 UTC m=+52.393132113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-47jkl" (UniqueName: "kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl") pod "network-check-target-r42mb" (UID: "3aa6f0a7-69fd-47a6-92c4-3bb5132afddc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:39.574184 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:39.573920    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs podName:15c1454e-931d-4ab2-8bcc-20db5808084e nodeName:}" failed. No retries permitted until 2026-07-09 03:43:55.573909087 +0000 UTC m=+52.393164849 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs") pod "network-metrics-daemon-czw4n" (UID: "15c1454e-931d-4ab2-8bcc-20db5808084e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:39.674837 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:39.674798    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:39.675035 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:39.674988    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:39.675085 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:39.675071    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret podName:56157167-4b57-4f59-9f0b-8925fea4b4b3 nodeName:}" failed. No retries permitted until 2026-07-09 03:43:55.675050047 +0000 UTC m=+52.494305821 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret") pod "global-pull-secret-syncer-pw8dq" (UID: "56157167-4b57-4f59-9f0b-8925fea4b4b3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:39.708580 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:39.708541    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:39.708781 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:39.708541    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:39.708781 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:39.708681    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:39.708781 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:39.708681    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:39.708955 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:39.708791    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:39.708955 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:39.708896    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:41.708278 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:41.708225    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:41.708278 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:41.708253    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:41.708278 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:41.708272    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:41.708951 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:41.708366    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:41.708951 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:41.708532    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:41.708951 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:41.708621    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:42.845469 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:42.845322    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg" event={"ID":"ba4762bd-3392-4767-9a9d-c377797a8aaa","Type":"ContainerStarted","Data":"b338ab38ae939f329cad401daddd3888d7de96f17bd9b1423dd98af8a5a6083d"}
Jul 09 03:43:42.856434 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:42.856377    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x655t" event={"ID":"c2af28c4-c7f0-4545-9b84-04f7ec3c9533","Type":"ContainerStarted","Data":"a3cba0bf3bb6a7185c380dc9ebf44c3fb8e83202b85000ba5abbaf8669977dd0"}
Jul 09 03:43:42.859790 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:42.859552    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fjm27" event={"ID":"9f8b0c0d-fb9f-4e43-8cda-ffeac9c9b389","Type":"ContainerStarted","Data":"eeee91b2877d09c88299f0580a98a20ab90e2ffcb1a979d0b5d5a4799c029d17"}
Jul 09 03:43:42.884936 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:42.884793    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-x655t" podStartSLOduration=5.810315554 podStartE2EDuration="19.884771957s" totalImagesPullingTime="14.074456403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:42.884179132 +0000 UTC m=+39.703434910" watchObservedRunningTime="2026-07-09 03:43:42.884771957 +0000 UTC m=+39.704027736"
Jul 09 03:43:42.901011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:42.900957    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fjm27" podStartSLOduration=1.368629267 podStartE2EDuration="19.900934105s" totalImagesPullingTime="18.532304838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:42.900046916 +0000 UTC m=+39.719302694" watchObservedRunningTime="2026-07-09 03:43:42.900934105 +0000 UTC m=+39.720189884"
Jul 09 03:43:43.708702 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.708515    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:43.708913 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.708583    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:43.708913 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:43.708766    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:43.708913 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:43.708853    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:43.708913 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.708607    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:43.709135 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:43.708944    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:43.864654 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.863952    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4bcnc" event={"ID":"2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f","Type":"ContainerStarted","Data":"4ff098f4f0e5835254789eedb58c67710ac2be90664db893fb55a1ec1aa4fbef"}
Jul 09 03:43:43.867538 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.867433    2582 generic.go:356] "Generic (PLEG): container finished" podID="be397f8d-fd42-44ea-b444-3530fe24606b" containerID="e4d72b194ed38a9881cce12a7682af959b5b0becc58b594ed78cf955c715499a" exitCode=0
Jul 09 03:43:43.867538 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.867477    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24dwk" event={"ID":"be397f8d-fd42-44ea-b444-3530fe24606b","Type":"ContainerDied","Data":"e4d72b194ed38a9881cce12a7682af959b5b0becc58b594ed78cf955c715499a"}
Jul 09 03:43:43.871163 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.871120    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 03:43:43.871524 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.871497    2582 generic.go:356] "Generic (PLEG): container finished" podID="98de9f69-0044-4adc-879b-5eb059d178ad" containerID="109a1322cc503d6545ca8be7b2452e3bfa512e7f52c1aec5b4dfd4a638197595" exitCode=1
Jul 09 03:43:43.871632 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.871563    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz" event={"ID":"98de9f69-0044-4adc-879b-5eb059d178ad","Type":"ContainerStarted","Data":"93fecaf6f25424e27071ea0683bd5dc48bfe8016a4f9a81864d66dd03250bd33"}
Jul 09 03:43:43.871684 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.871633    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz" event={"ID":"98de9f69-0044-4adc-879b-5eb059d178ad","Type":"ContainerStarted","Data":"c1ec6be415125baaca50b64d00fbf116a36649767697e5c3dee3d71f668a9d68"}
Jul 09 03:43:43.871684 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.871672    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz" event={"ID":"98de9f69-0044-4adc-879b-5eb059d178ad","Type":"ContainerStarted","Data":"cd1ce024f20862fd2e79a207962edbc7ce83f047a2d4d660931abe2a3fd864a3"}
Jul 09 03:43:43.871782 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.871711    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz" event={"ID":"98de9f69-0044-4adc-879b-5eb059d178ad","Type":"ContainerStarted","Data":"1fddef03723ddfc851dd8f6f5e2888973f3936a0a9577217648dae1d8d56411a"}
Jul 09 03:43:43.871782 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.871749    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz" event={"ID":"98de9f69-0044-4adc-879b-5eb059d178ad","Type":"ContainerDied","Data":"109a1322cc503d6545ca8be7b2452e3bfa512e7f52c1aec5b4dfd4a638197595"}
Jul 09 03:43:43.871893 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.871797    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz" event={"ID":"98de9f69-0044-4adc-879b-5eb059d178ad","Type":"ContainerStarted","Data":"25c21800bfe5835ac44279da9ed952b76865851c8fce2dcc3103e16719ada699"}
Jul 09 03:43:43.873395 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.873368    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2wsms" event={"ID":"fd5a5f67-7cb7-4196-b7ef-8f3227eed070","Type":"ContainerStarted","Data":"822b417c29a34ccbcc8ac6897387c086ba0b1cb218abcff233e0eb1d1f6bfb89"}
Jul 09 03:43:43.875007 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.874982    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-942xr" event={"ID":"ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72","Type":"ContainerStarted","Data":"956a30024927597d34d392f154311b60cc4cb906be70078d6fd1846222b7bccc"}
Jul 09 03:43:43.892620 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.892566    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4bcnc" podStartSLOduration=2.296934199 podStartE2EDuration="20.892548974s" totalImagesPullingTime="18.595614775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:43.878575215 +0000 UTC m=+40.697830994" watchObservedRunningTime="2026-07-09 03:43:43.892548974 +0000 UTC m=+40.711804755"
Jul 09 03:43:43.892935 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.892897    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2wsms" podStartSLOduration=2.356682822 podStartE2EDuration="20.89288864s" totalImagesPullingTime="18.536205818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:43.892455399 +0000 UTC m=+40.711711180" watchObservedRunningTime="2026-07-09 03:43:43.89288864 +0000 UTC m=+40.712144419"
Jul 09 03:43:43.910960 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:43.910910    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-942xr" podStartSLOduration=2.033700023 podStartE2EDuration="20.910893713s" totalImagesPullingTime="18.87719369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:43.910641532 +0000 UTC m=+40.729897310" watchObservedRunningTime="2026-07-09 03:43:43.910893713 +0000 UTC m=+40.730149490"
Jul 09 03:43:44.009595 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.009567    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:44.010187 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.010168    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:44.058559 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.058519    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:44.116969 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.116927    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:44.291246 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.291222    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 03:43:44.402759 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.402721    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:44.621493 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.621395    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T03:43:44.291240369Z","UUID":"49fa17aa-e10c-49bc-a922-258d369ff13f","Handler":null,"Name":"","Endpoint":""}
Jul 09 03:43:44.623025 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.623002    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 03:43:44.623160 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.623040    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 03:43:44.879538 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.879449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg" event={"ID":"ba4762bd-3392-4767-9a9d-c377797a8aaa","Type":"ContainerStarted","Data":"eadf682cb47f82a9ddf4531472e206552a60f6a5840dd6db1a6467bb91656870"}
Jul 09 03:43:44.881213 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.881181    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sr9tx" event={"ID":"75bfd683-da7d-4dfc-8179-884e0455367e","Type":"ContainerStarted","Data":"a2901b43afa8132947ac65154527c3bc35b16606e132b355ece1d54dd123f4b4"}
Jul 09 03:43:44.881594 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.881571    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:44.882176 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.882153    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4bcnc"
Jul 09 03:43:44.895635 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:44.895587    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sr9tx" podStartSLOduration=3.264690903 podStartE2EDuration="21.895570263s" totalImagesPullingTime="18.63087936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:44.895314942 +0000 UTC m=+41.714570722" watchObservedRunningTime="2026-07-09 03:43:44.895570263 +0000 UTC m=+41.714826041"
Jul 09 03:43:45.013953 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:45.013919    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:45.067361 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:45.067321    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:45.645274 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:45.645200    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:45.708226 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:45.708203    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:45.708350 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:45.708202    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:45.708350 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:45.708307    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:45.708472 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:45.708410    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:45.708472 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:45.708202    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:45.708540 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:45.708526    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:45.738747 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:45.738692    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:45.886339 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:45.886292    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg" event={"ID":"ba4762bd-3392-4767-9a9d-c377797a8aaa","Type":"ContainerStarted","Data":"5b870875c4d489b504887c0dc1a23d292a9ce6926b116a83d8188a283b37c3db"}
Jul 09 03:43:45.890182 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:45.890156    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 03:43:45.890628 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:45.890602    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz" event={"ID":"98de9f69-0044-4adc-879b-5eb059d178ad","Type":"ContainerStarted","Data":"839435b3cacf34fd7b2adcc1cbc594a7fb37267691268ff11e5fb4584a2ecbd9"}
Jul 09 03:43:46.234160 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:46.234120    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:47.079739 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:47.079690    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:47.180859 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:47.180827    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:47.298879 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:47.298833    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:47.349116 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:47.349019    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:47.707892 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:47.707786    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:47.708162 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:47.707786    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:47.708162 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:47.707936    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:47.708162 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:47.707786    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:47.708162 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:47.707994    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:47.708162 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:47.708117    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:47.938667 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:47.938630    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:48.010747 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.010648    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjntg" podStartSLOduration=8.376271212 podStartE2EDuration="25.010625903s" totalImagesPullingTime="16.634354691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:45.904934793 +0000 UTC m=+42.724190568" watchObservedRunningTime="2026-07-09 03:43:48.010625903 +0000 UTC m=+44.829881681"
Jul 09 03:43:48.011372 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.011348    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jcnfn"]
Jul 09 03:43:48.018190 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.018157    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.020704 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.020676    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 03:43:48.021221 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.021199    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 03:43:48.021221 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.021213    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bhfm5\""
Jul 09 03:43:48.021397 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.021198    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 03:43:48.022178 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.022159    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 03:43:48.022280 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.022195    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 03:43:48.022280 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.022223    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 03:43:48.034334 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.034305    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:48.138020 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.137989    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f76f7924-686f-446e-905b-b343df740a64-node-exporter-wtmp\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.138020 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.138022    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f76f7924-686f-446e-905b-b343df740a64-metrics-client-ca\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.138540 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.138046    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f76f7924-686f-446e-905b-b343df740a64-root\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.138540 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.138069    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f76f7924-686f-446e-905b-b343df740a64-node-exporter-accelerators-collector-config\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.138540 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.138157    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f76f7924-686f-446e-905b-b343df740a64-sys\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.138540 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.138204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f76f7924-686f-446e-905b-b343df740a64-node-exporter-textfile\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.138540 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.138256    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f76f7924-686f-446e-905b-b343df740a64-node-exporter-tls\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.138540 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.138283    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f76f7924-686f-446e-905b-b343df740a64-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.138540 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.138321    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmkvp\" (UniqueName: \"kubernetes.io/projected/f76f7924-686f-446e-905b-b343df740a64-kube-api-access-gmkvp\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.206348 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.206310    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:48.239070 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239031    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmkvp\" (UniqueName: \"kubernetes.io/projected/f76f7924-686f-446e-905b-b343df740a64-kube-api-access-gmkvp\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239257 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239124    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f76f7924-686f-446e-905b-b343df740a64-node-exporter-wtmp\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239257 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239153    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f76f7924-686f-446e-905b-b343df740a64-metrics-client-ca\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239257 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239185    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f76f7924-686f-446e-905b-b343df740a64-root\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239257 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239211    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f76f7924-686f-446e-905b-b343df740a64-node-exporter-accelerators-collector-config\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239257 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239241    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f76f7924-686f-446e-905b-b343df740a64-sys\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239525 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239276    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f76f7924-686f-446e-905b-b343df740a64-node-exporter-textfile\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239580 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f76f7924-686f-446e-905b-b343df740a64-node-exporter-tls\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239580 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239567    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f76f7924-686f-446e-905b-b343df740a64-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239684 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239588    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f76f7924-686f-446e-905b-b343df740a64-node-exporter-textfile\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239684 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239345    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f76f7924-686f-446e-905b-b343df740a64-sys\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239684 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239283    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f76f7924-686f-446e-905b-b343df740a64-root\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239684 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239308    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f76f7924-686f-446e-905b-b343df740a64-node-exporter-wtmp\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239907 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239889    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f76f7924-686f-446e-905b-b343df740a64-metrics-client-ca\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.239977 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.239947    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f76f7924-686f-446e-905b-b343df740a64-node-exporter-accelerators-collector-config\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.243853 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.243828    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f76f7924-686f-446e-905b-b343df740a64-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.243999 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.243974    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f76f7924-686f-446e-905b-b343df740a64-node-exporter-tls\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.247317 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.247290    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmkvp\" (UniqueName: \"kubernetes.io/projected/f76f7924-686f-446e-905b-b343df740a64-kube-api-access-gmkvp\") pod \"node-exporter-jcnfn\" (UID: \"f76f7924-686f-446e-905b-b343df740a64\") " pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.306194 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.306150    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:48.329027 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.328991    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jcnfn"
Jul 09 03:43:48.405716 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.405682    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:48.506143 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:48.506094    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:49.012453 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:49.012424    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf76f7924_686f_446e_905b_b343df740a64.slice/crio-87ce12df6a61aa644c0b1f79436fed3d966c5d439aa766ae3947040a32996992 WatchSource:0}: Error finding container 87ce12df6a61aa644c0b1f79436fed3d966c5d439aa766ae3947040a32996992: Status 404 returned error can't find the container with id 87ce12df6a61aa644c0b1f79436fed3d966c5d439aa766ae3947040a32996992
Jul 09 03:43:49.061635 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.061274    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:49.151032 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.150782    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:49.243328 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.243291    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:49.462792 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.462751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:49.708262 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.708182    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:49.708262 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.708228    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:49.708465 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.708182    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:49.708465 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:49.708328    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:49.708465 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:49.708390    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:49.708623 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:49.708471    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:49.901299 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.901247    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jcnfn" event={"ID":"f76f7924-686f-446e-905b-b343df740a64","Type":"ContainerStarted","Data":"87ce12df6a61aa644c0b1f79436fed3d966c5d439aa766ae3947040a32996992"}
Jul 09 03:43:49.903486 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.903452    2582 generic.go:356] "Generic (PLEG): container finished" podID="be397f8d-fd42-44ea-b444-3530fe24606b" containerID="6ad0b97bd4dc2c226206ad137fe297df106c13aa198f0acef0644920b7abcb0b" exitCode=0
Jul 09 03:43:49.903653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.903545    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24dwk" event={"ID":"be397f8d-fd42-44ea-b444-3530fe24606b","Type":"ContainerDied","Data":"6ad0b97bd4dc2c226206ad137fe297df106c13aa198f0acef0644920b7abcb0b"}
Jul 09 03:43:49.907705 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.907670    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 03:43:49.908160 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.908107    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz" event={"ID":"98de9f69-0044-4adc-879b-5eb059d178ad","Type":"ContainerStarted","Data":"10f11f08061908e30055d4d6dddbf4388863a68c60377ea6ab1241d229d3f329"}
Jul 09 03:43:49.908415 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.908397    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:49.908595 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.908569    2582 scope.go:117] "RemoveContainer" containerID="109a1322cc503d6545ca8be7b2452e3bfa512e7f52c1aec5b4dfd4a638197595"
Jul 09 03:43:49.926025 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:49.925991    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:50.128610 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:50.128430    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:50.229051 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:50.229013    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:50.914713 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:50.914634    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 03:43:50.915042 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:50.915018    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz" event={"ID":"98de9f69-0044-4adc-879b-5eb059d178ad","Type":"ContainerStarted","Data":"1cb0d41ed41d37df2831c7b8934a989632c58e4468025f0bcbc550558981d48e"}
Jul 09 03:43:50.915232 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:50.915215    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:50.915320 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:50.915299    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:50.916777 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:50.916747    2582 generic.go:356] "Generic (PLEG): container finished" podID="f76f7924-686f-446e-905b-b343df740a64" containerID="fe2a2a819d0e79201340559ad15e3373d5f803496c3080c6e4db672edc6fff0e" exitCode=0
Jul 09 03:43:50.916892 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:50.916783    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jcnfn" event={"ID":"f76f7924-686f-446e-905b-b343df740a64","Type":"ContainerDied","Data":"fe2a2a819d0e79201340559ad15e3373d5f803496c3080c6e4db672edc6fff0e"}
Jul 09 03:43:50.930368 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:50.930339    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:43:50.947040 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:50.946990    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz" podStartSLOduration=9.2181417 podStartE2EDuration="27.946973531s" totalImagesPullingTime="18.728831831s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:50.946332894 +0000 UTC m=+47.765588690" watchObservedRunningTime="2026-07-09 03:43:50.946973531 +0000 UTC m=+47.766229309"
Jul 09 03:43:50.994743 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:50.994709    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:51.071096 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.071065    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:51.093692 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.093649    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pw8dq"]
Jul 09 03:43:51.093857 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.093793    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:51.093963 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:51.093938    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:51.094308 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.094288    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r42mb"]
Jul 09 03:43:51.094390 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.094380    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:51.094459 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:51.094444    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:51.108723 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.108704    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-czw4n"]
Jul 09 03:43:51.108844 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.108832    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:51.108988 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:51.108966    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:51.139347 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.139309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:51.590454 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.590425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:51.921953 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.921907    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jcnfn" event={"ID":"f76f7924-686f-446e-905b-b343df740a64","Type":"ContainerStarted","Data":"460c7917df442f0592807b59eba012d502d1b470c0ef714fd89d8f0efb950d99"}
Jul 09 03:43:51.922113 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.921987    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jcnfn" event={"ID":"f76f7924-686f-446e-905b-b343df740a64","Type":"ContainerStarted","Data":"8de957a56d0fe349089ab178872f4911e1e69398e2746a63da38fd63594f8155"}
Jul 09 03:43:51.924375 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.924346    2582 generic.go:356] "Generic (PLEG): container finished" podID="be397f8d-fd42-44ea-b444-3530fe24606b" containerID="56575dafa1ab078069c0ce197f665d441e75913d7f5ff1ae872ebb322cc91155" exitCode=0
Jul 09 03:43:51.924521 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.924425    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24dwk" event={"ID":"be397f8d-fd42-44ea-b444-3530fe24606b","Type":"ContainerDied","Data":"56575dafa1ab078069c0ce197f665d441e75913d7f5ff1ae872ebb322cc91155"}
Jul 09 03:43:51.944271 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:51.944230    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jcnfn" podStartSLOduration=3.435183925 podStartE2EDuration="4.944215947s" totalImagesPullingTime="1.364914683s" totalInitContainerRuntime="144.117339ms" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:43:51.94312905 +0000 UTC m=+48.762384827" watchObservedRunningTime="2026-07-09 03:43:51.944215947 +0000 UTC m=+48.763471752"
Jul 09 03:43:52.708888 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:52.708694    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:52.709268 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:52.708728    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:52.709268 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:52.708735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:52.709268 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:52.709029    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:52.709268 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:52.709133    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:52.709268 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:52.709200    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:52.880898 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:52.880851    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:52.955078 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:52.955046    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:53.086267 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:53.086236    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:53.323974 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:53.323937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:53.422198 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:53.422164    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:53.523373 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:53.523331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:53.932002 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:53.931967    2582 generic.go:356] "Generic (PLEG): container finished" podID="be397f8d-fd42-44ea-b444-3530fe24606b" containerID="bd859575b2857c36050961c505b72db77915a29d5b37a99a89a4cef3aa64096e" exitCode=0
Jul 09 03:43:53.932520 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:53.932023    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24dwk" event={"ID":"be397f8d-fd42-44ea-b444-3530fe24606b","Type":"ContainerDied","Data":"bd859575b2857c36050961c505b72db77915a29d5b37a99a89a4cef3aa64096e"}
Jul 09 03:43:54.291991 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:54.291954    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:54.708742 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:54.708658    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:54.708742 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:54.708680    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:54.709063 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:54.708668    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:54.709063 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:54.708788    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r42mb" podUID="3aa6f0a7-69fd-47a6-92c4-3bb5132afddc"
Jul 09 03:43:54.709063 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:54.708834    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8dq" podUID="56157167-4b57-4f59-9f0b-8925fea4b4b3"
Jul 09 03:43:54.709063 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:54.708954    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czw4n" podUID="15c1454e-931d-4ab2-8bcc-20db5808084e"
Jul 09 03:43:55.600205 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.600164    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:55.600641 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.600216    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47jkl\" (UniqueName: \"kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl\") pod \"network-check-target-r42mb\" (UID: \"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc\") " pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:55.600641 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:55.600384    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:55.600641 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:55.600394    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 03:43:55.600641 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:55.600411    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 03:43:55.600641 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:55.600426    2582 projected.go:196] Error preparing data for projected volume kube-api-access-47jkl for pod openshift-network-diagnostics/network-check-target-r42mb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:55.600641 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:55.600457    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs podName:15c1454e-931d-4ab2-8bcc-20db5808084e nodeName:}" failed. No retries permitted until 2026-07-09 03:44:27.600438052 +0000 UTC m=+84.419693831 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs") pod "network-metrics-daemon-czw4n" (UID: "15c1454e-931d-4ab2-8bcc-20db5808084e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 03:43:55.600641 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:55.600473    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl podName:3aa6f0a7-69fd-47a6-92c4-3bb5132afddc nodeName:}" failed. No retries permitted until 2026-07-09 03:44:27.60046704 +0000 UTC m=+84.419722796 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-47jkl" (UniqueName: "kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl") pod "network-check-target-r42mb" (UID: "3aa6f0a7-69fd-47a6-92c4-3bb5132afddc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 03:43:55.601663 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.601638    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-164.ec2.internal" event="NodeReady"
Jul 09 03:43:55.601802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.601778    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 03:43:55.647532 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.647482    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xv4td"]
Jul 09 03:43:55.662915 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.662772    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n29bj"]
Jul 09 03:43:55.663290 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.662982    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xv4td"
Jul 09 03:43:55.666047 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.665815    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k4rlp\""
Jul 09 03:43:55.666047 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.665835    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 03:43:55.666047 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.665992    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 03:43:55.666284 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.666116    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 03:43:55.688448 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.688417    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xv4td"]
Jul 09 03:43:55.688606 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.688462    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n29bj"]
Jul 09 03:43:55.688606 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.688540    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.692414 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.692387    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 03:43:55.692566 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.692386    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-phtqr\""
Jul 09 03:43:55.692566 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.692524    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 03:43:55.693117 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.693001    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 03:43:55.693117 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.693093    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 03:43:55.701018 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.700989    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:55.701170 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:55.701113    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:55.701232 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:43:55.701171    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret podName:56157167-4b57-4f59-9f0b-8925fea4b4b3 nodeName:}" failed. No retries permitted until 2026-07-09 03:44:27.701154187 +0000 UTC m=+84.520409947 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret") pod "global-pull-secret-syncer-pw8dq" (UID: "56157167-4b57-4f59-9f0b-8925fea4b4b3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 03:43:55.766655 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.766615    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4gjlw"]
Jul 09 03:43:55.789129 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.789092    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4gjlw"]
Jul 09 03:43:55.789298 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.789247    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:55.792490 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.792467    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 03:43:55.792642 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.792469    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hwlx6\""
Jul 09 03:43:55.792642 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.792580    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 03:43:55.801466 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.801440    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/478e4ae2-fc93-4443-93f2-83f1208cb291-cert\") pod \"ingress-canary-xv4td\" (UID: \"478e4ae2-fc93-4443-93f2-83f1208cb291\") " pod="openshift-ingress-canary/ingress-canary-xv4td"
Jul 09 03:43:55.801599 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.801477    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe660e68-2687-427e-94c8-2d8fc456a42c-data-volume\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.801599 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.801499    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btnxl\" (UniqueName: \"kubernetes.io/projected/fe660e68-2687-427e-94c8-2d8fc456a42c-kube-api-access-btnxl\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.801599 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.801522    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe660e68-2687-427e-94c8-2d8fc456a42c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.801808 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.801614    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpxht\" (UniqueName: \"kubernetes.io/projected/478e4ae2-fc93-4443-93f2-83f1208cb291-kube-api-access-kpxht\") pod \"ingress-canary-xv4td\" (UID: \"478e4ae2-fc93-4443-93f2-83f1208cb291\") " pod="openshift-ingress-canary/ingress-canary-xv4td"
Jul 09 03:43:55.801808 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.801638    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fe660e68-2687-427e-94c8-2d8fc456a42c-crio-socket\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.801808 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.801686    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fe660e68-2687-427e-94c8-2d8fc456a42c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.902485 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902379    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe660e68-2687-427e-94c8-2d8fc456a42c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.902485 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902425    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-metrics-tls\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:55.902485 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902448    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqjpv\" (UniqueName: \"kubernetes.io/projected/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-kube-api-access-vqjpv\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:55.902766 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902605    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpxht\" (UniqueName: \"kubernetes.io/projected/478e4ae2-fc93-4443-93f2-83f1208cb291-kube-api-access-kpxht\") pod \"ingress-canary-xv4td\" (UID: \"478e4ae2-fc93-4443-93f2-83f1208cb291\") " pod="openshift-ingress-canary/ingress-canary-xv4td"
Jul 09 03:43:55.902766 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902648    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fe660e68-2687-427e-94c8-2d8fc456a42c-crio-socket\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.902766 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902714    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-tmp-dir\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:55.902766 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902753    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fe660e68-2687-427e-94c8-2d8fc456a42c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.902987 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902798    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/478e4ae2-fc93-4443-93f2-83f1208cb291-cert\") pod \"ingress-canary-xv4td\" (UID: \"478e4ae2-fc93-4443-93f2-83f1208cb291\") " pod="openshift-ingress-canary/ingress-canary-xv4td"
Jul 09 03:43:55.902987 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902832    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe660e68-2687-427e-94c8-2d8fc456a42c-data-volume\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.902987 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902855    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fe660e68-2687-427e-94c8-2d8fc456a42c-crio-socket\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.902987 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902884    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btnxl\" (UniqueName: \"kubernetes.io/projected/fe660e68-2687-427e-94c8-2d8fc456a42c-kube-api-access-btnxl\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.902987 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.902911    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-config-volume\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:55.903400 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.903336    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe660e68-2687-427e-94c8-2d8fc456a42c-data-volume\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.903515 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.903490    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fe660e68-2687-427e-94c8-2d8fc456a42c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.907323 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.907298    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fe660e68-2687-427e-94c8-2d8fc456a42c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.907465 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.907354    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/478e4ae2-fc93-4443-93f2-83f1208cb291-cert\") pod \"ingress-canary-xv4td\" (UID: \"478e4ae2-fc93-4443-93f2-83f1208cb291\") " pod="openshift-ingress-canary/ingress-canary-xv4td"
Jul 09 03:43:55.911329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.911301    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btnxl\" (UniqueName: \"kubernetes.io/projected/fe660e68-2687-427e-94c8-2d8fc456a42c-kube-api-access-btnxl\") pod \"insights-runtime-extractor-n29bj\" (UID: \"fe660e68-2687-427e-94c8-2d8fc456a42c\") " pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:55.911444 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.911303    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpxht\" (UniqueName: \"kubernetes.io/projected/478e4ae2-fc93-4443-93f2-83f1208cb291-kube-api-access-kpxht\") pod \"ingress-canary-xv4td\" (UID: \"478e4ae2-fc93-4443-93f2-83f1208cb291\") " pod="openshift-ingress-canary/ingress-canary-xv4td"
Jul 09 03:43:55.975177 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.975144    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xv4td"
Jul 09 03:43:56.000011 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:55.999973    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n29bj"
Jul 09 03:43:56.003934 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.003907    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-config-volume\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:56.004107 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.003943    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-metrics-tls\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:56.004107 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.003969    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqjpv\" (UniqueName: \"kubernetes.io/projected/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-kube-api-access-vqjpv\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:56.004107 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.004036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-tmp-dir\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:56.004478 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.004456    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-tmp-dir\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:56.004753 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.004724    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-config-volume\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:56.006701 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.006669    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-metrics-tls\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:56.013806 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.013777    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqjpv\" (UniqueName: \"kubernetes.io/projected/b9f541c8-fdea-4bf7-b3f2-61ae2262afc2-kube-api-access-vqjpv\") pod \"dns-default-4gjlw\" (UID: \"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2\") " pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:56.098777 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.098734    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:43:56.201504 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.201417    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n29bj"]
Jul 09 03:43:56.205005 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.204977    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xv4td"]
Jul 09 03:43:56.205423 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:56.205394    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe660e68_2687_427e_94c8_2d8fc456a42c.slice/crio-6e852651cb872404883e04472b23747cd136a5c7ba207d66aadaee3385d970fd WatchSource:0}: Error finding container 6e852651cb872404883e04472b23747cd136a5c7ba207d66aadaee3385d970fd: Status 404 returned error can't find the container with id 6e852651cb872404883e04472b23747cd136a5c7ba207d66aadaee3385d970fd
Jul 09 03:43:56.208482 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:56.208261    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478e4ae2_fc93_4443_93f2_83f1208cb291.slice/crio-23b88fb23d846cb7a69604a98bc02158961291b1751b59d48a3d146516ff430e WatchSource:0}: Error finding container 23b88fb23d846cb7a69604a98bc02158961291b1751b59d48a3d146516ff430e: Status 404 returned error can't find the container with id 23b88fb23d846cb7a69604a98bc02158961291b1751b59d48a3d146516ff430e
Jul 09 03:43:56.279333 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.279293    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4gjlw"]
Jul 09 03:43:56.284090 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:43:56.284064    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f541c8_fdea_4bf7_b3f2_61ae2262afc2.slice/crio-cb1f250e7cbea1dac3a4cb75a6776d2a46232a329fe05ab2c6fc9979227181ca WatchSource:0}: Error finding container cb1f250e7cbea1dac3a4cb75a6776d2a46232a329fe05ab2c6fc9979227181ca: Status 404 returned error can't find the container with id cb1f250e7cbea1dac3a4cb75a6776d2a46232a329fe05ab2c6fc9979227181ca
Jul 09 03:43:56.707796 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.707758    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:43:56.708285 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.707759    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:43:56.708285 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.707759    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:43:56.710535 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.710514    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 03:43:56.711655 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.711633    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 03:43:56.711775 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.711680    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-54tq2\""
Jul 09 03:43:56.711836 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.711681    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 03:43:56.712072 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.711981    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ch8mv\""
Jul 09 03:43:56.712213 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.712093    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 03:43:56.944358 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.943919    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n29bj" event={"ID":"fe660e68-2687-427e-94c8-2d8fc456a42c","Type":"ContainerStarted","Data":"6e852651cb872404883e04472b23747cd136a5c7ba207d66aadaee3385d970fd"}
Jul 09 03:43:56.947557 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.947470    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xv4td" event={"ID":"478e4ae2-fc93-4443-93f2-83f1208cb291","Type":"ContainerStarted","Data":"23b88fb23d846cb7a69604a98bc02158961291b1751b59d48a3d146516ff430e"}
Jul 09 03:43:56.949985 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:56.949908    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4gjlw" event={"ID":"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2","Type":"ContainerStarted","Data":"cb1f250e7cbea1dac3a4cb75a6776d2a46232a329fe05ab2c6fc9979227181ca"}
Jul 09 03:43:57.028123 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:57.028026    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:57.131305 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:57.131262    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:58.741985 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:58.741949    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:58.795690 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:58.795653    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:43:59.990657 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:43:59.990617    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:00.042697 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:00.042658    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:03.971639 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:03.971611    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n29bj" event={"ID":"fe660e68-2687-427e-94c8-2d8fc456a42c","Type":"ContainerStarted","Data":"a28dfd2e13ab123f57f959a1a1d7a14ef7eb6a3059e91ae00cc8e7b1da8732c2"}
Jul 09 03:44:03.973719 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:03.973369    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xv4td" event={"ID":"478e4ae2-fc93-4443-93f2-83f1208cb291","Type":"ContainerStarted","Data":"1e4f6d1449bf947273f4074bdeb0a71ed8a5e86e544558297d61a54feb8fc14e"}
Jul 09 03:44:03.989059 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:03.988782    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xv4td" podStartSLOduration=1.341450561 podStartE2EDuration="8.988761168s" totalImagesPullingTime="7.647310607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:44:03.987768587 +0000 UTC m=+60.807024380" watchObservedRunningTime="2026-07-09 03:44:03.988761168 +0000 UTC m=+60.808016947"
Jul 09 03:44:04.344439 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:04.344396    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:04.445455 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:04.445368    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:04.545440 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:04.545393    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:04.978684 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:04.978644    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n29bj" event={"ID":"fe660e68-2687-427e-94c8-2d8fc456a42c","Type":"ContainerStarted","Data":"718a90d45937d231d8ba83aed949383b20b1d42248257a79a61a249828e2545e"}
Jul 09 03:44:04.981403 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:04.981377    2582 generic.go:356] "Generic (PLEG): container finished" podID="be397f8d-fd42-44ea-b444-3530fe24606b" containerID="45032bfe2192ef817a966094f9908ff55b5bd19415800a069c38ab21121f5148" exitCode=0
Jul 09 03:44:04.981540 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:04.981453    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24dwk" event={"ID":"be397f8d-fd42-44ea-b444-3530fe24606b","Type":"ContainerDied","Data":"45032bfe2192ef817a966094f9908ff55b5bd19415800a069c38ab21121f5148"}
Jul 09 03:44:04.983227 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:04.983202    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4gjlw" event={"ID":"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2","Type":"ContainerStarted","Data":"6e173e11e455774800cf681bc527e0d703bed7eb99573b174964ba2cd80ddb15"}
Jul 09 03:44:04.983329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:04.983264    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4gjlw" event={"ID":"b9f541c8-fdea-4bf7-b3f2-61ae2262afc2","Type":"ContainerStarted","Data":"d8a8df0d14795ed3199925e8a06b96384355acb8f592c8e84b809acd9cf2b82a"}
Jul 09 03:44:05.025299 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:05.025243    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4gjlw" podStartSLOduration=2.453662648 podStartE2EDuration="10.025223006s" totalImagesPullingTime="7.571560358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:44:05.024188915 +0000 UTC m=+61.843444690" watchObservedRunningTime="2026-07-09 03:44:05.025223006 +0000 UTC m=+61.844478780"
Jul 09 03:44:05.989634 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:05.989593    2582 generic.go:356] "Generic (PLEG): container finished" podID="be397f8d-fd42-44ea-b444-3530fe24606b" containerID="278867aa677207c266c738d34d524c694b60fe70f70fb83f8a4a9eff5af827ed" exitCode=0
Jul 09 03:44:05.990108 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:05.989669    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24dwk" event={"ID":"be397f8d-fd42-44ea-b444-3530fe24606b","Type":"ContainerDied","Data":"278867aa677207c266c738d34d524c694b60fe70f70fb83f8a4a9eff5af827ed"}
Jul 09 03:44:05.990108 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:05.989991    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:44:07.999709 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:07.999671    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24dwk" event={"ID":"be397f8d-fd42-44ea-b444-3530fe24606b","Type":"ContainerStarted","Data":"910aa6092046ac67ccbd8ed403b17179d5e191a60ee4cb58c298468627ab2702"}
Jul 09 03:44:08.001571 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:08.001545    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n29bj" event={"ID":"fe660e68-2687-427e-94c8-2d8fc456a42c","Type":"ContainerStarted","Data":"ac01d0a859967d199e9eb41271f25e9b9b6aad4dd365485251fbc574617997eb"}
Jul 09 03:44:08.026284 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:08.024596    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-24dwk" podStartSLOduration=8.55528516 podStartE2EDuration="45.02457525s" totalImagesPullingTime="36.46929009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:44:08.022752946 +0000 UTC m=+64.842008724" watchObservedRunningTime="2026-07-09 03:44:08.02457525 +0000 UTC m=+64.843831028"
Jul 09 03:44:08.041644 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:08.041588    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n29bj" podStartSLOduration=2.283382921 podStartE2EDuration="13.041568622s" totalImagesPullingTime="10.758185701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:44:08.040781877 +0000 UTC m=+64.860037655" watchObservedRunningTime="2026-07-09 03:44:08.041568622 +0000 UTC m=+64.860824400"
Jul 09 03:44:08.146010 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:08.145976    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:08.196092 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:08.196056    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:13.747031 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:13.746991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:13.795442 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:13.795402    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:15.995577 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:15.995546    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4gjlw"
Jul 09 03:44:16.439281 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:16.439248    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:22.937583 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:22.937556    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2pkz"
Jul 09 03:44:23.048873 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:23.048836    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:23.112549 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:23.112510    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:27.652742 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.652703    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:44:27.653358 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.652751    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47jkl\" (UniqueName: \"kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl\") pod \"network-check-target-r42mb\" (UID: \"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc\") " pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:44:27.655924 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.655899    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 03:44:27.656071 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.655901    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 03:44:27.665585 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.665557    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 03:44:27.666217 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.666191    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c1454e-931d-4ab2-8bcc-20db5808084e-metrics-certs\") pod \"network-metrics-daemon-czw4n\" (UID: \"15c1454e-931d-4ab2-8bcc-20db5808084e\") " pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:44:27.675856 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.675832    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47jkl\" (UniqueName: \"kubernetes.io/projected/3aa6f0a7-69fd-47a6-92c4-3bb5132afddc-kube-api-access-47jkl\") pod \"network-check-target-r42mb\" (UID: \"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc\") " pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:44:27.753108 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.753060    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:44:27.756140 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.756114    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 03:44:27.766621 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.766588    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56157167-4b57-4f59-9f0b-8925fea4b4b3-original-pull-secret\") pod \"global-pull-secret-syncer-pw8dq\" (UID: \"56157167-4b57-4f59-9f0b-8925fea4b4b3\") " pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:44:27.923585 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.923512    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ch8mv\""
Jul 09 03:44:27.930088 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.930061    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-54tq2\""
Jul 09 03:44:27.931071 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.931048    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:44:27.934357 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.934331    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8dq"
Jul 09 03:44:27.938004 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:27.937981    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czw4n"
Jul 09 03:44:28.091500 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:28.091458    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pw8dq"]
Jul 09 03:44:28.095192 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:44:28.095149    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56157167_4b57_4f59_9f0b_8925fea4b4b3.slice/crio-8216194113ae8e5ab2fae12edc1067a71956968859c3476ec27e12af1642d3de WatchSource:0}: Error finding container 8216194113ae8e5ab2fae12edc1067a71956968859c3476ec27e12af1642d3de: Status 404 returned error can't find the container with id 8216194113ae8e5ab2fae12edc1067a71956968859c3476ec27e12af1642d3de
Jul 09 03:44:28.319356 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:28.319318    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r42mb"]
Jul 09 03:44:28.320264 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:28.320234    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-czw4n"]
Jul 09 03:44:28.322506 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:44:28.322476    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c1454e_931d_4ab2_8bcc_20db5808084e.slice/crio-624a07c622946075594b4bbe7f62decf714179aa8f88d86d1133909e1ac14fd0 WatchSource:0}: Error finding container 624a07c622946075594b4bbe7f62decf714179aa8f88d86d1133909e1ac14fd0: Status 404 returned error can't find the container with id 624a07c622946075594b4bbe7f62decf714179aa8f88d86d1133909e1ac14fd0
Jul 09 03:44:28.323316 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:44:28.323294    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aa6f0a7_69fd_47a6_92c4_3bb5132afddc.slice/crio-f843281667903fb143b04e26f79216b29c55747df50dabb1194e435e83bb9a25 WatchSource:0}: Error finding container f843281667903fb143b04e26f79216b29c55747df50dabb1194e435e83bb9a25: Status 404 returned error can't find the container with id f843281667903fb143b04e26f79216b29c55747df50dabb1194e435e83bb9a25
Jul 09 03:44:29.077740 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:29.077620    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-czw4n" event={"ID":"15c1454e-931d-4ab2-8bcc-20db5808084e","Type":"ContainerStarted","Data":"624a07c622946075594b4bbe7f62decf714179aa8f88d86d1133909e1ac14fd0"}
Jul 09 03:44:29.081516 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:29.081481    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pw8dq" event={"ID":"56157167-4b57-4f59-9f0b-8925fea4b4b3","Type":"ContainerStarted","Data":"8216194113ae8e5ab2fae12edc1067a71956968859c3476ec27e12af1642d3de"}
Jul 09 03:44:29.083085 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:29.083054    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r42mb" event={"ID":"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc","Type":"ContainerStarted","Data":"f843281667903fb143b04e26f79216b29c55747df50dabb1194e435e83bb9a25"}
Jul 09 03:44:30.089086 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:30.089034    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-czw4n" event={"ID":"15c1454e-931d-4ab2-8bcc-20db5808084e","Type":"ContainerStarted","Data":"a496afcbb742250453878a33b4faecf5d115a8fc00657ffc3fec55ee062bc959"}
Jul 09 03:44:30.089551 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:30.089116    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-czw4n" event={"ID":"15c1454e-931d-4ab2-8bcc-20db5808084e","Type":"ContainerStarted","Data":"2e3338a619e3ea00d9615154ac2c648b5eab4179e489a03f3817eb6204e03f85"}
Jul 09 03:44:30.104953 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:30.104891    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-czw4n" podStartSLOduration=65.802585456 podStartE2EDuration="1m7.104846345s" totalImagesPullingTime="1.302260889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:44:30.103629206 +0000 UTC m=+86.922884985" watchObservedRunningTime="2026-07-09 03:44:30.104846345 +0000 UTC m=+86.924102127"
Jul 09 03:44:30.187016 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:30.186979    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:30.239180 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:30.239141    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:30.297014 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:30.296882    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:44:34.106884 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:34.106823    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pw8dq" event={"ID":"56157167-4b57-4f59-9f0b-8925fea4b4b3","Type":"ContainerStarted","Data":"f7f12f47a96aa6e20cdbb1c130163f20fc9773894adfac4c6d11b558e084fd3e"}
Jul 09 03:44:34.108427 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:34.108400    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r42mb" event={"ID":"3aa6f0a7-69fd-47a6-92c4-3bb5132afddc","Type":"ContainerStarted","Data":"e3a9fc4a86e7e1753bc295263e3fe69b00428e1d6b466c1d3d6f24b6f113b20d"}
Jul 09 03:44:34.108561 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:34.108534    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:44:34.123350 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:34.123294    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pw8dq" podStartSLOduration=65.475133196 podStartE2EDuration="1m11.123276057s" totalImagesPullingTime="5.648142861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:44:34.123104375 +0000 UTC m=+90.942360152" watchObservedRunningTime="2026-07-09 03:44:34.123276057 +0000 UTC m=+90.942531837"
Jul 09 03:44:34.136954 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:44:34.136904    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r42mb" podStartSLOduration=65.728242575 podStartE2EDuration="1m11.136886926s" totalImagesPullingTime="5.408644351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:44:34.136605784 +0000 UTC m=+90.955861564" watchObservedRunningTime="2026-07-09 03:44:34.136886926 +0000 UTC m=+90.956142728"
Jul 09 03:45:05.114053 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:05.114016    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r42mb"
Jul 09 03:45:07.283369 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:07.283336    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:07.783378 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:07.783340    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:12.368161 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.368110    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"]
Jul 09 03:45:12.371004 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.370987    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.374254 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.374231    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 09 03:45:12.374774 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.374759    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 09 03:45:12.374836 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.374792    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 09 03:45:12.375052 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.375034    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bftn6\""
Jul 09 03:45:12.375133 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.375071    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 09 03:45:12.375133 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.375077    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 09 03:45:12.383072 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.383046    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 09 03:45:12.385215 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.385189    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"]
Jul 09 03:45:12.454858 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.454811    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgd7l\" (UniqueName: \"kubernetes.io/projected/5042b609-c946-4f78-bb43-b34d20a70c57-kube-api-access-pgd7l\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.454858 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.454855    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-federate-client-tls\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.455110 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.454926    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.455110 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.455001    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5042b609-c946-4f78-bb43-b34d20a70c57-telemeter-trusted-ca-bundle\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.455110 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.455037    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-secret-telemeter-client\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.455110 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.455054    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5042b609-c946-4f78-bb43-b34d20a70c57-metrics-client-ca\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.455110 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.455073    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-telemeter-client-tls\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.455261 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.455132    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5042b609-c946-4f78-bb43-b34d20a70c57-serving-certs-ca-bundle\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.556052 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.556018    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.556293 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.556059    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5042b609-c946-4f78-bb43-b34d20a70c57-telemeter-trusted-ca-bundle\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.556293 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.556195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-secret-telemeter-client\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.556293 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.556236    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5042b609-c946-4f78-bb43-b34d20a70c57-metrics-client-ca\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.556293 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.556271    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-telemeter-client-tls\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.556468 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.556304    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5042b609-c946-4f78-bb43-b34d20a70c57-serving-certs-ca-bundle\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.556468 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.556337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgd7l\" (UniqueName: \"kubernetes.io/projected/5042b609-c946-4f78-bb43-b34d20a70c57-kube-api-access-pgd7l\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.556468 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.556363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-federate-client-tls\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.557126 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.557088    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5042b609-c946-4f78-bb43-b34d20a70c57-metrics-client-ca\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.557254 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.557219    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5042b609-c946-4f78-bb43-b34d20a70c57-telemeter-trusted-ca-bundle\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.557367 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.557347    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5042b609-c946-4f78-bb43-b34d20a70c57-serving-certs-ca-bundle\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.559100 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.559071    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-secret-telemeter-client\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.559202 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.559192    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-federate-client-tls\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.559242 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.559201    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.559393 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.559369    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5042b609-c946-4f78-bb43-b34d20a70c57-telemeter-client-tls\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.564449 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.564430    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgd7l\" (UniqueName: \"kubernetes.io/projected/5042b609-c946-4f78-bb43-b34d20a70c57-kube-api-access-pgd7l\") pod \"telemeter-client-869fd9cd76-mpqxt\" (UID: \"5042b609-c946-4f78-bb43-b34d20a70c57\") " pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.681691 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.681589    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"
Jul 09 03:45:12.824909 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:12.824859    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-869fd9cd76-mpqxt"]
Jul 09 03:45:12.828406 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:45:12.828377    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5042b609_c946_4f78_bb43_b34d20a70c57.slice/crio-0e0b2b03185a3951ef526b7b792a310e7125cd378591989cdab2fe37fd8a3232 WatchSource:0}: Error finding container 0e0b2b03185a3951ef526b7b792a310e7125cd378591989cdab2fe37fd8a3232: Status 404 returned error can't find the container with id 0e0b2b03185a3951ef526b7b792a310e7125cd378591989cdab2fe37fd8a3232
Jul 09 03:45:13.240659 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:13.240625    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt" event={"ID":"5042b609-c946-4f78-bb43-b34d20a70c57","Type":"ContainerStarted","Data":"0e0b2b03185a3951ef526b7b792a310e7125cd378591989cdab2fe37fd8a3232"}
Jul 09 03:45:15.248789 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:15.248755    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt" event={"ID":"5042b609-c946-4f78-bb43-b34d20a70c57","Type":"ContainerStarted","Data":"65a853e3d0a51a920774599d3d9d2f2d60d5c4988c71c8710036c157c3c2cf4e"}
Jul 09 03:45:17.257271 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:17.257234    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt" event={"ID":"5042b609-c946-4f78-bb43-b34d20a70c57","Type":"ContainerStarted","Data":"49ceea679c886c48ad12e4e0f16b84bff15cec8cdd899fbfa81d036b1af12983"}
Jul 09 03:45:17.257666 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:17.257322    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt" event={"ID":"5042b609-c946-4f78-bb43-b34d20a70c57","Type":"ContainerStarted","Data":"4d9b8955cfe6d0c5c0fcb0138bdae0fe81e665a60279261352057cca153f4def"}
Jul 09 03:45:17.279547 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:17.279061    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-869fd9cd76-mpqxt" podStartSLOduration=1.775507508 podStartE2EDuration="5.279040546s" totalImagesPullingTime="3.503533038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:45:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:45:17.278645921 +0000 UTC m=+134.097901711" watchObservedRunningTime="2026-07-09 03:45:17.279040546 +0000 UTC m=+134.098296325"
Jul 09 03:45:18.338395 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:18.338354    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:18.393778 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:18.393745    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:22.496395 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:22.496358    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:22.609911 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:22.609856    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:24.430307 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:24.430272    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:24.481675 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:24.481643    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:25.027729 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:25.027698    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:34.472917 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:34.472835    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:34.574321 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:34.574286    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:34.725309 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:34.725224    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:34.826157 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:34.826124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:34.877458 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:34.877421    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:43.750269 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:43.750233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:43.851788 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:43.851752    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:44.485125 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:44.485088    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:44.585658 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:44.585626    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:47.472077 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:47.472041    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:47.524755 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:47.524726    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:45:52.158341 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:45:52.158307    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:46:20.256875 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:20.256837    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:46:27.657732 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:27.657695    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:46:27.752793 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:27.752753    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:46:28.255798 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.255762    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"]
Jul 09 03:46:28.258955 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.258938    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:28.261583 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.261557    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 03:46:28.261731 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.261586    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 03:46:28.261731 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.261711    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t6txf\""
Jul 09 03:46:28.266963 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.266937    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"]
Jul 09 03:46:28.278087 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.278060    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:28.278240 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.278104    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9525c\" (UniqueName: \"kubernetes.io/projected/2a7ca677-5281-4546-b045-1640561ae436-kube-api-access-9525c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:28.278240 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.278124    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:28.379475 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.379431    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:28.379656 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.379498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:28.379656 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.379539    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9525c\" (UniqueName: \"kubernetes.io/projected/2a7ca677-5281-4546-b045-1640561ae436-kube-api-access-9525c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:28.379824 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.379804    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:28.379858 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.379834    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:28.390568 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.390540    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9525c\" (UniqueName: \"kubernetes.io/projected/2a7ca677-5281-4546-b045-1640561ae436-kube-api-access-9525c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:28.568448 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.568414    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:28.695164 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:28.695132    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"]
Jul 09 03:46:28.698294 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:46:28.698267    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a7ca677_5281_4546_b045_1640561ae436.slice/crio-d63f7e5078a25cbb6fbe151913639acc0a86cb5ca012a1b957da2fd9cb11b45a WatchSource:0}: Error finding container d63f7e5078a25cbb6fbe151913639acc0a86cb5ca012a1b957da2fd9cb11b45a: Status 404 returned error can't find the container with id d63f7e5078a25cbb6fbe151913639acc0a86cb5ca012a1b957da2fd9cb11b45a
Jul 09 03:46:29.485669 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:29.485635    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w" event={"ID":"2a7ca677-5281-4546-b045-1640561ae436","Type":"ContainerStarted","Data":"d63f7e5078a25cbb6fbe151913639acc0a86cb5ca012a1b957da2fd9cb11b45a"}
Jul 09 03:46:36.511819 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:36.511780    2582 generic.go:356] "Generic (PLEG): container finished" podID="2a7ca677-5281-4546-b045-1640561ae436" containerID="8a15dd7e860f8ea70cf051f2b2556875d730105a07386d5474a3c70f0cbe14a1" exitCode=0
Jul 09 03:46:36.512218 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:36.511886    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w" event={"ID":"2a7ca677-5281-4546-b045-1640561ae436","Type":"ContainerDied","Data":"8a15dd7e860f8ea70cf051f2b2556875d730105a07386d5474a3c70f0cbe14a1"}
Jul 09 03:46:39.523418 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:39.523388    2582 generic.go:356] "Generic (PLEG): container finished" podID="2a7ca677-5281-4546-b045-1640561ae436" containerID="fae7a08706230c94a53b22414d2f911d5e692c2550f7ac8671f7381c8a0e4fed" exitCode=0
Jul 09 03:46:39.523831 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:39.523455    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w" event={"ID":"2a7ca677-5281-4546-b045-1640561ae436","Type":"ContainerDied","Data":"fae7a08706230c94a53b22414d2f911d5e692c2550f7ac8671f7381c8a0e4fed"}
Jul 09 03:46:47.554371 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:47.554329    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w" event={"ID":"2a7ca677-5281-4546-b045-1640561ae436","Type":"ContainerStarted","Data":"45d2fa353ce233eab64bd6dcdab72bf6183302b463ada7700a51f74256fac2d4"}
Jul 09 03:46:47.570787 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:47.570733    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w" podStartSLOduration=1.939398144 podStartE2EDuration="19.570713965s" totalImagesPullingTime="17.631315821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:46:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:46:47.569966422 +0000 UTC m=+224.389222199" watchObservedRunningTime="2026-07-09 03:46:47.570713965 +0000 UTC m=+224.389969741"
Jul 09 03:46:48.559315 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:48.559281    2582 generic.go:356] "Generic (PLEG): container finished" podID="2a7ca677-5281-4546-b045-1640561ae436" containerID="45d2fa353ce233eab64bd6dcdab72bf6183302b463ada7700a51f74256fac2d4" exitCode=0
Jul 09 03:46:48.559710 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:48.559340    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w" event={"ID":"2a7ca677-5281-4546-b045-1640561ae436","Type":"ContainerDied","Data":"45d2fa353ce233eab64bd6dcdab72bf6183302b463ada7700a51f74256fac2d4"}
Jul 09 03:46:49.688117 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:49.688086    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:49.735888 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:49.735837    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9525c\" (UniqueName: \"kubernetes.io/projected/2a7ca677-5281-4546-b045-1640561ae436-kube-api-access-9525c\") pod \"2a7ca677-5281-4546-b045-1640561ae436\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") "
Jul 09 03:46:49.736073 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:49.735917    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-util\") pod \"2a7ca677-5281-4546-b045-1640561ae436\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") "
Jul 09 03:46:49.736073 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:49.735964    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-bundle\") pod \"2a7ca677-5281-4546-b045-1640561ae436\" (UID: \"2a7ca677-5281-4546-b045-1640561ae436\") "
Jul 09 03:46:49.736603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:49.736572    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-bundle" (OuterVolumeSpecName: "bundle") pod "2a7ca677-5281-4546-b045-1640561ae436" (UID: "2a7ca677-5281-4546-b045-1640561ae436"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:46:49.738152 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:49.738123    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a7ca677-5281-4546-b045-1640561ae436-kube-api-access-9525c" (OuterVolumeSpecName: "kube-api-access-9525c") pod "2a7ca677-5281-4546-b045-1640561ae436" (UID: "2a7ca677-5281-4546-b045-1640561ae436"). InnerVolumeSpecName "kube-api-access-9525c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:46:49.740163 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:49.740140    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-util" (OuterVolumeSpecName: "util") pod "2a7ca677-5281-4546-b045-1640561ae436" (UID: "2a7ca677-5281-4546-b045-1640561ae436"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:46:49.836666 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:49.836585    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-bundle\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:46:49.836666 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:49.836618    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9525c\" (UniqueName: \"kubernetes.io/projected/2a7ca677-5281-4546-b045-1640561ae436-kube-api-access-9525c\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:46:49.836666 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:49.836655    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a7ca677-5281-4546-b045-1640561ae436-util\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:46:50.567905 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:50.567854    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w" event={"ID":"2a7ca677-5281-4546-b045-1640561ae436","Type":"ContainerDied","Data":"d63f7e5078a25cbb6fbe151913639acc0a86cb5ca012a1b957da2fd9cb11b45a"}
Jul 09 03:46:50.568136 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:50.567925    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d63f7e5078a25cbb6fbe151913639acc0a86cb5ca012a1b957da2fd9cb11b45a"
Jul 09 03:46:50.568136 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:50.567878    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txl9w"
Jul 09 03:46:54.272723 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:54.272683    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:46:55.056274 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:55.056230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:46:56.234501 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.234468    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c"]
Jul 09 03:46:56.234914 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.234697    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a7ca677-5281-4546-b045-1640561ae436" containerName="util"
Jul 09 03:46:56.234914 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.234707    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a7ca677-5281-4546-b045-1640561ae436" containerName="util"
Jul 09 03:46:56.234914 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.234716    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a7ca677-5281-4546-b045-1640561ae436" containerName="extract"
Jul 09 03:46:56.234914 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.234723    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a7ca677-5281-4546-b045-1640561ae436" containerName="extract"
Jul 09 03:46:56.234914 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.234736    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a7ca677-5281-4546-b045-1640561ae436" containerName="pull"
Jul 09 03:46:56.234914 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.234741    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a7ca677-5281-4546-b045-1640561ae436" containerName="pull"
Jul 09 03:46:56.234914 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.234782    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a7ca677-5281-4546-b045-1640561ae436" containerName="extract"
Jul 09 03:46:56.236500 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.236485    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c"
Jul 09 03:46:56.238944 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.238918    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 09 03:46:56.239082 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.239005    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-g55p4\""
Jul 09 03:46:56.239144 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.239100    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 09 03:46:56.249193 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.249158    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c"]
Jul 09 03:46:56.277752 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.277714    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/663d87ab-3e61-4550-8b78-8109525ad3a1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjc5c\" (UID: \"663d87ab-3e61-4550-8b78-8109525ad3a1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c"
Jul 09 03:46:56.277752 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.277754    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgbpv\" (UniqueName: \"kubernetes.io/projected/663d87ab-3e61-4550-8b78-8109525ad3a1-kube-api-access-mgbpv\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjc5c\" (UID: \"663d87ab-3e61-4550-8b78-8109525ad3a1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c"
Jul 09 03:46:56.378160 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.378109    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/663d87ab-3e61-4550-8b78-8109525ad3a1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjc5c\" (UID: \"663d87ab-3e61-4550-8b78-8109525ad3a1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c"
Jul 09 03:46:56.378160 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.378162    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgbpv\" (UniqueName: \"kubernetes.io/projected/663d87ab-3e61-4550-8b78-8109525ad3a1-kube-api-access-mgbpv\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjc5c\" (UID: \"663d87ab-3e61-4550-8b78-8109525ad3a1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c"
Jul 09 03:46:56.378521 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.378497    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/663d87ab-3e61-4550-8b78-8109525ad3a1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjc5c\" (UID: \"663d87ab-3e61-4550-8b78-8109525ad3a1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c"
Jul 09 03:46:56.389253 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.389225    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgbpv\" (UniqueName: \"kubernetes.io/projected/663d87ab-3e61-4550-8b78-8109525ad3a1-kube-api-access-mgbpv\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjc5c\" (UID: \"663d87ab-3e61-4550-8b78-8109525ad3a1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c"
Jul 09 03:46:56.544990 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.544957    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c"
Jul 09 03:46:56.678177 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:56.678152    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c"]
Jul 09 03:46:56.681109 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:46:56.681075    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663d87ab_3e61_4550_8b78_8109525ad3a1.slice/crio-a4de8f5dd88e2d3c061073d04ad8a7f9aec05cf8a82e7d706fda2ed6cebc4982 WatchSource:0}: Error finding container a4de8f5dd88e2d3c061073d04ad8a7f9aec05cf8a82e7d706fda2ed6cebc4982: Status 404 returned error can't find the container with id a4de8f5dd88e2d3c061073d04ad8a7f9aec05cf8a82e7d706fda2ed6cebc4982
Jul 09 03:46:57.596313 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:57.596262    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c" event={"ID":"663d87ab-3e61-4550-8b78-8109525ad3a1","Type":"ContainerStarted","Data":"a4de8f5dd88e2d3c061073d04ad8a7f9aec05cf8a82e7d706fda2ed6cebc4982"}
Jul 09 03:46:57.878371 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:57.878282    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:46:57.933143 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:57.933111    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:46:59.246179 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.246145    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:46:59.345329 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.345288    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:46:59.605491 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.605455    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c" event={"ID":"663d87ab-3e61-4550-8b78-8109525ad3a1","Type":"ContainerStarted","Data":"96ff8e52458ca1f93a5736f2668ce658fbe102636159de4991106aed3c2e3ab4"}
Jul 09 03:46:59.628487 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.628431    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjc5c" podStartSLOduration=1.7280779910000001 podStartE2EDuration="3.628414146s" totalImagesPullingTime="1.900336155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:46:59.626187702 +0000 UTC m=+236.445443479" watchObservedRunningTime="2026-07-09 03:46:59.628414146 +0000 UTC m=+236.447669922"
Jul 09 03:46:59.630594 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.630560    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-v2v28"]
Jul 09 03:46:59.632717 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.632697    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28"
Jul 09 03:46:59.636176 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.636154    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qnlwk\""
Jul 09 03:46:59.636282 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.636219    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 03:46:59.636328 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.636154    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 03:46:59.643326 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.643302    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-v2v28"]
Jul 09 03:46:59.702027 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.701987    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e490906-6538-47e6-ac1d-fa1ce64a6fa7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-v2v28\" (UID: \"4e490906-6538-47e6-ac1d-fa1ce64a6fa7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28"
Jul 09 03:46:59.702208 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.702064    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65sxx\" (UniqueName: \"kubernetes.io/projected/4e490906-6538-47e6-ac1d-fa1ce64a6fa7-kube-api-access-65sxx\") pod \"cert-manager-webhook-6486ffd997-v2v28\" (UID: \"4e490906-6538-47e6-ac1d-fa1ce64a6fa7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28"
Jul 09 03:46:59.803016 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.802971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65sxx\" (UniqueName: \"kubernetes.io/projected/4e490906-6538-47e6-ac1d-fa1ce64a6fa7-kube-api-access-65sxx\") pod \"cert-manager-webhook-6486ffd997-v2v28\" (UID: \"4e490906-6538-47e6-ac1d-fa1ce64a6fa7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28"
Jul 09 03:46:59.803193 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.803030    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e490906-6538-47e6-ac1d-fa1ce64a6fa7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-v2v28\" (UID: \"4e490906-6538-47e6-ac1d-fa1ce64a6fa7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28"
Jul 09 03:46:59.816970 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.816938    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e490906-6538-47e6-ac1d-fa1ce64a6fa7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-v2v28\" (UID: \"4e490906-6538-47e6-ac1d-fa1ce64a6fa7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28"
Jul 09 03:46:59.824901 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.824853    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65sxx\" (UniqueName: \"kubernetes.io/projected/4e490906-6538-47e6-ac1d-fa1ce64a6fa7-kube-api-access-65sxx\") pod \"cert-manager-webhook-6486ffd997-v2v28\" (UID: \"4e490906-6538-47e6-ac1d-fa1ce64a6fa7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28"
Jul 09 03:46:59.952435 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:46:59.952345    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28"
Jul 09 03:47:00.022158 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.022123    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:00.093210 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.093186    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-v2v28"]
Jul 09 03:47:00.095649 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:47:00.095611    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e490906_6538_47e6_ac1d_fa1ce64a6fa7.slice/crio-8ef698565385deafce8de6f27d930dfc824d1f46db4f4fabf77f6a586f373a5a WatchSource:0}: Error finding container 8ef698565385deafce8de6f27d930dfc824d1f46db4f4fabf77f6a586f373a5a: Status 404 returned error can't find the container with id 8ef698565385deafce8de6f27d930dfc824d1f46db4f4fabf77f6a586f373a5a
Jul 09 03:47:00.112081 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.112048    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:00.531130 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.531100    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"]
Jul 09 03:47:00.534267 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.534246    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:00.537027 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.537002    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 03:47:00.537140 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.537002    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 03:47:00.537140 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.537091    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t6txf\""
Jul 09 03:47:00.542327 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.542301    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"]
Jul 09 03:47:00.608985 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.608957    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:00.609181 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.608994    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p5t8\" (UniqueName: \"kubernetes.io/projected/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-kube-api-access-6p5t8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:00.609181 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.609091    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:00.610029 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.610006    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28" event={"ID":"4e490906-6538-47e6-ac1d-fa1ce64a6fa7","Type":"ContainerStarted","Data":"8ef698565385deafce8de6f27d930dfc824d1f46db4f4fabf77f6a586f373a5a"}
Jul 09 03:47:00.709969 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.709925    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:00.710159 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.709985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:00.710159 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.710019    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p5t8\" (UniqueName: \"kubernetes.io/projected/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-kube-api-access-6p5t8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:00.710403 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.710379    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:00.710463 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.710390    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:00.718104 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.718069    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p5t8\" (UniqueName: \"kubernetes.io/projected/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-kube-api-access-6p5t8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:00.844276 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.844193    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:00.982996 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:00.982957    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"]
Jul 09 03:47:00.986534 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:47:00.986503    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c4e2cb_03be_4e8a_a9fe_96d66ed904ca.slice/crio-c7f6540fee340da63970b751d5e334813ad0bfe8225201cfe3befe95e702d077 WatchSource:0}: Error finding container c7f6540fee340da63970b751d5e334813ad0bfe8225201cfe3befe95e702d077: Status 404 returned error can't find the container with id c7f6540fee340da63970b751d5e334813ad0bfe8225201cfe3befe95e702d077
Jul 09 03:47:01.616561 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:01.616325    2582 generic.go:356] "Generic (PLEG): container finished" podID="73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" containerID="4e1985deca002e63e9cd848198e200be36b341e47aa476252987b4058174a03b" exitCode=0
Jul 09 03:47:01.616561 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:01.616417    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw" event={"ID":"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca","Type":"ContainerDied","Data":"4e1985deca002e63e9cd848198e200be36b341e47aa476252987b4058174a03b"}
Jul 09 03:47:01.616561 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:01.616460    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw" event={"ID":"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca","Type":"ContainerStarted","Data":"c7f6540fee340da63970b751d5e334813ad0bfe8225201cfe3befe95e702d077"}
Jul 09 03:47:01.652932 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:01.652890    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:02.062884 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:02.062835    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:02.853525 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:02.853490    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf"]
Jul 09 03:47:02.856897 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:02.856857    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf"
Jul 09 03:47:02.862039 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:02.862013    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-z6mzs\""
Jul 09 03:47:02.866990 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:02.866968    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf"]
Jul 09 03:47:02.927314 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:02.927269    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/273a0a34-325f-42fe-867e-eb92e184e607-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pzfjf\" (UID: \"273a0a34-325f-42fe-867e-eb92e184e607\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf"
Jul 09 03:47:02.927478 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:02.927363    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqzqm\" (UniqueName: \"kubernetes.io/projected/273a0a34-325f-42fe-867e-eb92e184e607-kube-api-access-bqzqm\") pod \"cert-manager-cainjector-559cdd8c87-pzfjf\" (UID: \"273a0a34-325f-42fe-867e-eb92e184e607\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf"
Jul 09 03:47:03.027980 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:03.027942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/273a0a34-325f-42fe-867e-eb92e184e607-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pzfjf\" (UID: \"273a0a34-325f-42fe-867e-eb92e184e607\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf"
Jul 09 03:47:03.028194 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:03.028004    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqzqm\" (UniqueName: \"kubernetes.io/projected/273a0a34-325f-42fe-867e-eb92e184e607-kube-api-access-bqzqm\") pod \"cert-manager-cainjector-559cdd8c87-pzfjf\" (UID: \"273a0a34-325f-42fe-867e-eb92e184e607\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf"
Jul 09 03:47:03.036598 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:03.036564    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/273a0a34-325f-42fe-867e-eb92e184e607-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pzfjf\" (UID: \"273a0a34-325f-42fe-867e-eb92e184e607\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf"
Jul 09 03:47:03.036758 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:03.036679    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqzqm\" (UniqueName: \"kubernetes.io/projected/273a0a34-325f-42fe-867e-eb92e184e607-kube-api-access-bqzqm\") pod \"cert-manager-cainjector-559cdd8c87-pzfjf\" (UID: \"273a0a34-325f-42fe-867e-eb92e184e607\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf"
Jul 09 03:47:03.167468 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:03.167384    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf"
Jul 09 03:47:04.242057 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:04.242027    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf"]
Jul 09 03:47:04.245055 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:47:04.245022    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod273a0a34_325f_42fe_867e_eb92e184e607.slice/crio-3432d84956be0573e05f6ecae539824b6725087c18092afd5be38ce383781418 WatchSource:0}: Error finding container 3432d84956be0573e05f6ecae539824b6725087c18092afd5be38ce383781418: Status 404 returned error can't find the container with id 3432d84956be0573e05f6ecae539824b6725087c18092afd5be38ce383781418
Jul 09 03:47:04.631432 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:04.631401    2582 generic.go:356] "Generic (PLEG): container finished" podID="73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" containerID="5c66963a84e890d4120b7c2f24e9ff8df24202904a1e18b245ac83700a63d824" exitCode=0
Jul 09 03:47:04.631629 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:04.631469    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw" event={"ID":"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca","Type":"ContainerDied","Data":"5c66963a84e890d4120b7c2f24e9ff8df24202904a1e18b245ac83700a63d824"}
Jul 09 03:47:04.633125 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:04.633097    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf" event={"ID":"273a0a34-325f-42fe-867e-eb92e184e607","Type":"ContainerStarted","Data":"e331dcf42d6783953a62cda5835ed1c692a9d32d89bfc8766ed99671f6f57d78"}
Jul 09 03:47:04.633256 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:04.633134    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf" event={"ID":"273a0a34-325f-42fe-867e-eb92e184e607","Type":"ContainerStarted","Data":"3432d84956be0573e05f6ecae539824b6725087c18092afd5be38ce383781418"}
Jul 09 03:47:04.634827 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:04.634793    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28" event={"ID":"4e490906-6538-47e6-ac1d-fa1ce64a6fa7","Type":"ContainerStarted","Data":"dad12ce68ada4f045543aa0d279061a31964069894ab49d096f1838e61e6dfbc"}
Jul 09 03:47:04.634923 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:04.634850    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28"
Jul 09 03:47:04.666352 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:04.665584    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28" podStartSLOduration=1.646827765 podStartE2EDuration="5.66556192s" totalImagesPullingTime="4.018734155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:47:04.664082861 +0000 UTC m=+241.483338639" watchObservedRunningTime="2026-07-09 03:47:04.66556192 +0000 UTC m=+241.484817698"
Jul 09 03:47:04.683536 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:04.683481    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pzfjf" podStartSLOduration=2.683463241 podStartE2EDuration="2.683463241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:47:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:47:04.68217471 +0000 UTC m=+241.501430488" watchObservedRunningTime="2026-07-09 03:47:04.683463241 +0000 UTC m=+241.502719017"
Jul 09 03:47:04.684502 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:04.684473    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:04.781929 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:04.781887    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:05.640478 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:05.640443    2582 generic.go:356] "Generic (PLEG): container finished" podID="73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" containerID="d306a668f27b766417e5c19198c1d0e482d3e70236a873101b5433b15849b453" exitCode=0
Jul 09 03:47:05.640842 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:05.640532    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw" event={"ID":"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca","Type":"ContainerDied","Data":"d306a668f27b766417e5c19198c1d0e482d3e70236a873101b5433b15849b453"}
Jul 09 03:47:06.766963 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:06.766937    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:06.859067 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:06.859029    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-bundle\") pod \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") "
Jul 09 03:47:06.859067 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:06.859077    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6p5t8\" (UniqueName: \"kubernetes.io/projected/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-kube-api-access-6p5t8\") pod \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") "
Jul 09 03:47:06.859331 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:06.859115    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-util\") pod \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\" (UID: \"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca\") "
Jul 09 03:47:06.859513 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:06.859483    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-bundle" (OuterVolumeSpecName: "bundle") pod "73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" (UID: "73c4e2cb-03be-4e8a-a9fe-96d66ed904ca"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:47:06.861213 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:06.861191    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-kube-api-access-6p5t8" (OuterVolumeSpecName: "kube-api-access-6p5t8") pod "73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" (UID: "73c4e2cb-03be-4e8a-a9fe-96d66ed904ca"). InnerVolumeSpecName "kube-api-access-6p5t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:47:06.863848 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:06.863813    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-util" (OuterVolumeSpecName: "util") pod "73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" (UID: "73c4e2cb-03be-4e8a-a9fe-96d66ed904ca"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:47:06.959929 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:06.959802    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-util\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:47:06.959929 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:06.959833    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-bundle\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:47:06.959929 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:06.959842    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p5t8\" (UniqueName: \"kubernetes.io/projected/73c4e2cb-03be-4e8a-a9fe-96d66ed904ca-kube-api-access-6p5t8\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:47:07.649444 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:07.649411    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw" event={"ID":"73c4e2cb-03be-4e8a-a9fe-96d66ed904ca","Type":"ContainerDied","Data":"c7f6540fee340da63970b751d5e334813ad0bfe8225201cfe3befe95e702d077"}
Jul 09 03:47:07.649444 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:07.649438    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkcscw"
Jul 09 03:47:07.649630 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:07.649458    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7f6540fee340da63970b751d5e334813ad0bfe8225201cfe3befe95e702d077"
Jul 09 03:47:08.861763 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:08.861732    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:09.538378 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.538341    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tbjrg"]
Jul 09 03:47:09.538610 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.538594    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" containerName="extract"
Jul 09 03:47:09.538653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.538613    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" containerName="extract"
Jul 09 03:47:09.538653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.538625    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" containerName="util"
Jul 09 03:47:09.538653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.538631    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" containerName="util"
Jul 09 03:47:09.538653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.538644    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" containerName="pull"
Jul 09 03:47:09.538653 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.538649    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" containerName="pull"
Jul 09 03:47:09.538841 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.538695    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="73c4e2cb-03be-4e8a-a9fe-96d66ed904ca" containerName="extract"
Jul 09 03:47:09.542296 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.542278    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tbjrg"
Jul 09 03:47:09.544582 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.544561    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-6nnx8\""
Jul 09 03:47:09.549642 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.549618    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tbjrg"]
Jul 09 03:47:09.681447 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.681394    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90625745-6f86-4076-bee2-c20224cc7770-bound-sa-token\") pod \"cert-manager-55b4d954d5-tbjrg\" (UID: \"90625745-6f86-4076-bee2-c20224cc7770\") " pod="cert-manager/cert-manager-55b4d954d5-tbjrg"
Jul 09 03:47:09.681447 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.681452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l97c\" (UniqueName: \"kubernetes.io/projected/90625745-6f86-4076-bee2-c20224cc7770-kube-api-access-2l97c\") pod \"cert-manager-55b4d954d5-tbjrg\" (UID: \"90625745-6f86-4076-bee2-c20224cc7770\") " pod="cert-manager/cert-manager-55b4d954d5-tbjrg"
Jul 09 03:47:09.782178 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.782130    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90625745-6f86-4076-bee2-c20224cc7770-bound-sa-token\") pod \"cert-manager-55b4d954d5-tbjrg\" (UID: \"90625745-6f86-4076-bee2-c20224cc7770\") " pod="cert-manager/cert-manager-55b4d954d5-tbjrg"
Jul 09 03:47:09.782178 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.782183    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l97c\" (UniqueName: \"kubernetes.io/projected/90625745-6f86-4076-bee2-c20224cc7770-kube-api-access-2l97c\") pod \"cert-manager-55b4d954d5-tbjrg\" (UID: \"90625745-6f86-4076-bee2-c20224cc7770\") " pod="cert-manager/cert-manager-55b4d954d5-tbjrg"
Jul 09 03:47:09.793672 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.793601    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90625745-6f86-4076-bee2-c20224cc7770-bound-sa-token\") pod \"cert-manager-55b4d954d5-tbjrg\" (UID: \"90625745-6f86-4076-bee2-c20224cc7770\") " pod="cert-manager/cert-manager-55b4d954d5-tbjrg"
Jul 09 03:47:09.793672 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.793667    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l97c\" (UniqueName: \"kubernetes.io/projected/90625745-6f86-4076-bee2-c20224cc7770-kube-api-access-2l97c\") pod \"cert-manager-55b4d954d5-tbjrg\" (UID: \"90625745-6f86-4076-bee2-c20224cc7770\") " pod="cert-manager/cert-manager-55b4d954d5-tbjrg"
Jul 09 03:47:09.851673 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.851631    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tbjrg"
Jul 09 03:47:09.981969 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:09.981902    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tbjrg"]
Jul 09 03:47:09.984479 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:47:09.984451    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90625745_6f86_4076_bee2_c20224cc7770.slice/crio-87a5c5ac913882d41f8e9ece379502c871a18287214d1850df041e7fcc1bfffb WatchSource:0}: Error finding container 87a5c5ac913882d41f8e9ece379502c871a18287214d1850df041e7fcc1bfffb: Status 404 returned error can't find the container with id 87a5c5ac913882d41f8e9ece379502c871a18287214d1850df041e7fcc1bfffb
Jul 09 03:47:10.643345 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:10.643317    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-v2v28"
Jul 09 03:47:10.661122 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:10.661087    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tbjrg" event={"ID":"90625745-6f86-4076-bee2-c20224cc7770","Type":"ContainerStarted","Data":"3f8819458bb13911ad00fb0057d0965c21aa1f9da7bc68c66655e00937fadd07"}
Jul 09 03:47:10.661122 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:10.661128    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tbjrg" event={"ID":"90625745-6f86-4076-bee2-c20224cc7770","Type":"ContainerStarted","Data":"87a5c5ac913882d41f8e9ece379502c871a18287214d1850df041e7fcc1bfffb"}
Jul 09 03:47:10.679082 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:10.679014    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-tbjrg" podStartSLOduration=1.678990254 podStartE2EDuration="1.678990254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:47:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:47:10.67809874 +0000 UTC m=+247.497354519" watchObservedRunningTime="2026-07-09 03:47:10.678990254 +0000 UTC m=+247.498246034"
Jul 09 03:47:12.353081 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:12.353045    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:13.554979 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:13.554943    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:14.023754 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.023679    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq"]
Jul 09 03:47:14.052726 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.052695    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq"]
Jul 09 03:47:14.052919 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.052791    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq"
Jul 09 03:47:14.055802 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.055774    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 09 03:47:14.056794 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.056772    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-54f7c\""
Jul 09 03:47:14.056913 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.056777    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 09 03:47:14.214226 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.214185    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-986b2\" (UniqueName: \"kubernetes.io/projected/334257fc-37cd-4f40-a3ef-6bcb256a8f72-kube-api-access-986b2\") pod \"openshift-lws-operator-fd8ccff4c-gmdqq\" (UID: \"334257fc-37cd-4f40-a3ef-6bcb256a8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq"
Jul 09 03:47:14.214397 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.214247    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/334257fc-37cd-4f40-a3ef-6bcb256a8f72-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gmdqq\" (UID: \"334257fc-37cd-4f40-a3ef-6bcb256a8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq"
Jul 09 03:47:14.315098 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.315058    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/334257fc-37cd-4f40-a3ef-6bcb256a8f72-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gmdqq\" (UID: \"334257fc-37cd-4f40-a3ef-6bcb256a8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq"
Jul 09 03:47:14.315295 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.315131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-986b2\" (UniqueName: \"kubernetes.io/projected/334257fc-37cd-4f40-a3ef-6bcb256a8f72-kube-api-access-986b2\") pod \"openshift-lws-operator-fd8ccff4c-gmdqq\" (UID: \"334257fc-37cd-4f40-a3ef-6bcb256a8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq"
Jul 09 03:47:14.315450 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.315429    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/334257fc-37cd-4f40-a3ef-6bcb256a8f72-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gmdqq\" (UID: \"334257fc-37cd-4f40-a3ef-6bcb256a8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq"
Jul 09 03:47:14.323503 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.323476    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-986b2\" (UniqueName: \"kubernetes.io/projected/334257fc-37cd-4f40-a3ef-6bcb256a8f72-kube-api-access-986b2\") pod \"openshift-lws-operator-fd8ccff4c-gmdqq\" (UID: \"334257fc-37cd-4f40-a3ef-6bcb256a8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq"
Jul 09 03:47:14.362464 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.362421    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq"
Jul 09 03:47:14.521663 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.521636    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq"]
Jul 09 03:47:14.524278 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:47:14.524246    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod334257fc_37cd_4f40_a3ef_6bcb256a8f72.slice/crio-5f00e2a64ce3a5d8e759d309f2d4b7c4e3e84614539a6537cc0616b536b22100 WatchSource:0}: Error finding container 5f00e2a64ce3a5d8e759d309f2d4b7c4e3e84614539a6537cc0616b536b22100: Status 404 returned error can't find the container with id 5f00e2a64ce3a5d8e759d309f2d4b7c4e3e84614539a6537cc0616b536b22100
Jul 09 03:47:14.675893 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:14.675804    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq" event={"ID":"334257fc-37cd-4f40-a3ef-6bcb256a8f72","Type":"ContainerStarted","Data":"5f00e2a64ce3a5d8e759d309f2d4b7c4e3e84614539a6537cc0616b536b22100"}
Jul 09 03:47:17.689828 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:17.689793    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq" event={"ID":"334257fc-37cd-4f40-a3ef-6bcb256a8f72","Type":"ContainerStarted","Data":"1a959322f1bb4866d5cf99e4f74920237e2e5bee1e6560e4d6b99d9b0e168a86"}
Jul 09 03:47:17.709622 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:17.709565    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gmdqq" podStartSLOduration=1.381950445 podStartE2EDuration="3.709546s" totalImagesPullingTime="2.327595555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:47:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:47:17.708647235 +0000 UTC m=+254.527903012" watchObservedRunningTime="2026-07-09 03:47:17.709546 +0000 UTC m=+254.528801779"
Jul 09 03:47:18.316478 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:18.316440    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:18.416334 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:18.416304    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:20.213541 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.213509    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"]
Jul 09 03:47:20.215725 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.215701    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:20.218156 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.218128    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 03:47:20.219205 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.219187    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t6txf\""
Jul 09 03:47:20.219262 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.219186    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 03:47:20.223629 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.223606    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"]
Jul 09 03:47:20.361141 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.361100    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:20.361328 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.361174    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:20.361328 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.361207    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64hp5\" (UniqueName: \"kubernetes.io/projected/5f1abcef-e762-40f2-bee3-b10bacdb6eab-kube-api-access-64hp5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:20.462151 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.462110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64hp5\" (UniqueName: \"kubernetes.io/projected/5f1abcef-e762-40f2-bee3-b10bacdb6eab-kube-api-access-64hp5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:20.462358 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.462182    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:20.462358 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.462247    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:20.462649 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.462626    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:20.462733 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.462646    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:20.469938 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.469880    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64hp5\" (UniqueName: \"kubernetes.io/projected/5f1abcef-e762-40f2-bee3-b10bacdb6eab-kube-api-access-64hp5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:20.524998 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.524971    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:20.525451 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.525419    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:20.651742 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.651719    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"]
Jul 09 03:47:20.654152 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:47:20.654116    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1abcef_e762_40f2_bee3_b10bacdb6eab.slice/crio-67e3b1320238a18f9cd19b00d0b31df73dd6eba6d185628537b5e4822831113b WatchSource:0}: Error finding container 67e3b1320238a18f9cd19b00d0b31df73dd6eba6d185628537b5e4822831113b: Status 404 returned error can't find the container with id 67e3b1320238a18f9cd19b00d0b31df73dd6eba6d185628537b5e4822831113b
Jul 09 03:47:20.702464 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:20.702436    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh" event={"ID":"5f1abcef-e762-40f2-bee3-b10bacdb6eab","Type":"ContainerStarted","Data":"67e3b1320238a18f9cd19b00d0b31df73dd6eba6d185628537b5e4822831113b"}
Jul 09 03:47:21.708324 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:21.708284    2582 generic.go:356] "Generic (PLEG): container finished" podID="5f1abcef-e762-40f2-bee3-b10bacdb6eab" containerID="699c88c1a2454e6261eddcfd95a51ac6fe2a3fff16a40ed9e26b8b516143319e" exitCode=0
Jul 09 03:47:21.713113 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:21.713086    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh" event={"ID":"5f1abcef-e762-40f2-bee3-b10bacdb6eab","Type":"ContainerDied","Data":"699c88c1a2454e6261eddcfd95a51ac6fe2a3fff16a40ed9e26b8b516143319e"}
Jul 09 03:47:23.720113 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:23.720076    2582 generic.go:356] "Generic (PLEG): container finished" podID="5f1abcef-e762-40f2-bee3-b10bacdb6eab" containerID="68c59fbef277fdff74c8971ec3c9a00c0246e47166d2835690964b8ad1d0b7ee" exitCode=0
Jul 09 03:47:23.720524 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:23.720158    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh" event={"ID":"5f1abcef-e762-40f2-bee3-b10bacdb6eab","Type":"ContainerDied","Data":"68c59fbef277fdff74c8971ec3c9a00c0246e47166d2835690964b8ad1d0b7ee"}
Jul 09 03:47:24.725742 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:24.725707    2582 generic.go:356] "Generic (PLEG): container finished" podID="5f1abcef-e762-40f2-bee3-b10bacdb6eab" containerID="aaee5e65124f0118e47cea8fb1d4309cf69949000ae382f9ea87ca930a3a4f44" exitCode=0
Jul 09 03:47:24.726138 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:24.725799    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh" event={"ID":"5f1abcef-e762-40f2-bee3-b10bacdb6eab","Type":"ContainerDied","Data":"aaee5e65124f0118e47cea8fb1d4309cf69949000ae382f9ea87ca930a3a4f44"}
Jul 09 03:47:25.044383 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:25.044349    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:25.144925 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:25.144858    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:25.863401 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:25.863371    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:26.000081 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:25.999983    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-util\") pod \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") "
Jul 09 03:47:26.000081 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:26.000032    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64hp5\" (UniqueName: \"kubernetes.io/projected/5f1abcef-e762-40f2-bee3-b10bacdb6eab-kube-api-access-64hp5\") pod \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") "
Jul 09 03:47:26.000319 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:26.000115    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-bundle\") pod \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\" (UID: \"5f1abcef-e762-40f2-bee3-b10bacdb6eab\") "
Jul 09 03:47:26.001003 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:26.000970    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-bundle" (OuterVolumeSpecName: "bundle") pod "5f1abcef-e762-40f2-bee3-b10bacdb6eab" (UID: "5f1abcef-e762-40f2-bee3-b10bacdb6eab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:47:26.002286 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:26.002254    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f1abcef-e762-40f2-bee3-b10bacdb6eab-kube-api-access-64hp5" (OuterVolumeSpecName: "kube-api-access-64hp5") pod "5f1abcef-e762-40f2-bee3-b10bacdb6eab" (UID: "5f1abcef-e762-40f2-bee3-b10bacdb6eab"). InnerVolumeSpecName "kube-api-access-64hp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:47:26.006224 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:26.006188    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-util" (OuterVolumeSpecName: "util") pod "5f1abcef-e762-40f2-bee3-b10bacdb6eab" (UID: "5f1abcef-e762-40f2-bee3-b10bacdb6eab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:47:26.101049 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:26.101012    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-bundle\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:47:26.101049 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:26.101041    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f1abcef-e762-40f2-bee3-b10bacdb6eab-util\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:47:26.101049 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:26.101050    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64hp5\" (UniqueName: \"kubernetes.io/projected/5f1abcef-e762-40f2-bee3-b10bacdb6eab-kube-api-access-64hp5\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:47:26.735668 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:26.735627    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh" event={"ID":"5f1abcef-e762-40f2-bee3-b10bacdb6eab","Type":"ContainerDied","Data":"67e3b1320238a18f9cd19b00d0b31df73dd6eba6d185628537b5e4822831113b"}
Jul 09 03:47:26.735668 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:26.735652    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2mngh"
Jul 09 03:47:26.735918 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:26.735684    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67e3b1320238a18f9cd19b00d0b31df73dd6eba6d185628537b5e4822831113b"
Jul 09 03:47:32.781009 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.780973    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:32.816426 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.816381    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"]
Jul 09 03:47:32.816711 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.816694    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f1abcef-e762-40f2-bee3-b10bacdb6eab" containerName="util"
Jul 09 03:47:32.816791 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.816715    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f1abcef-e762-40f2-bee3-b10bacdb6eab" containerName="util"
Jul 09 03:47:32.816791 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.816727    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f1abcef-e762-40f2-bee3-b10bacdb6eab" containerName="extract"
Jul 09 03:47:32.816791 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.816736    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f1abcef-e762-40f2-bee3-b10bacdb6eab" containerName="extract"
Jul 09 03:47:32.816791 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.816749    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f1abcef-e762-40f2-bee3-b10bacdb6eab" containerName="pull"
Jul 09 03:47:32.816791 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.816760    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f1abcef-e762-40f2-bee3-b10bacdb6eab" containerName="pull"
Jul 09 03:47:32.817063 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.816831    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f1abcef-e762-40f2-bee3-b10bacdb6eab" containerName="extract"
Jul 09 03:47:32.819032 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.819009    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:32.822917 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.822895    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-688t6\""
Jul 09 03:47:32.823050 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.822892    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 09 03:47:32.823050 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.822930    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 09 03:47:32.823050 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.822930    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 09 03:47:32.828434 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.828408    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"]
Jul 09 03:47:32.855361 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.855325    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:32.953563 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.953526    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/62a1507d-4169-4cc1-a312-088304e18bac-metrics-cert\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:32.953563 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.953561    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62a1507d-4169-4cc1-a312-088304e18bac-cert\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:32.953809 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.953601    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgv25\" (UniqueName: \"kubernetes.io/projected/62a1507d-4169-4cc1-a312-088304e18bac-kube-api-access-wgv25\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:32.953809 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:32.953692    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/62a1507d-4169-4cc1-a312-088304e18bac-manager-config\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:33.054821 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:33.054732    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/62a1507d-4169-4cc1-a312-088304e18bac-manager-config\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:33.054821 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:33.054780    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/62a1507d-4169-4cc1-a312-088304e18bac-metrics-cert\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:33.054821 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:33.054798    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62a1507d-4169-4cc1-a312-088304e18bac-cert\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:33.055117 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:33.054836    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgv25\" (UniqueName: \"kubernetes.io/projected/62a1507d-4169-4cc1-a312-088304e18bac-kube-api-access-wgv25\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:33.055554 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:33.055529    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/62a1507d-4169-4cc1-a312-088304e18bac-manager-config\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:33.057542 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:33.057515    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62a1507d-4169-4cc1-a312-088304e18bac-cert\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:33.057638 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:33.057515    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/62a1507d-4169-4cc1-a312-088304e18bac-metrics-cert\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:33.064687 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:33.064665    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgv25\" (UniqueName: \"kubernetes.io/projected/62a1507d-4169-4cc1-a312-088304e18bac-kube-api-access-wgv25\") pod \"lws-controller-manager-7f5f9c4b6d-x4tgr\" (UID: \"62a1507d-4169-4cc1-a312-088304e18bac\") " pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:33.128585 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:33.128550    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:33.258191 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:33.258009    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"]
Jul 09 03:47:33.260590 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:47:33.260566    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62a1507d_4169_4cc1_a312_088304e18bac.slice/crio-125acf7d7ebd1f7e325f09245aec422c634feac974dee40b29bea624a5df7ea8 WatchSource:0}: Error finding container 125acf7d7ebd1f7e325f09245aec422c634feac974dee40b29bea624a5df7ea8: Status 404 returned error can't find the container with id 125acf7d7ebd1f7e325f09245aec422c634feac974dee40b29bea624a5df7ea8
Jul 09 03:47:33.764444 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:33.764399    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr" event={"ID":"62a1507d-4169-4cc1-a312-088304e18bac","Type":"ContainerStarted","Data":"125acf7d7ebd1f7e325f09245aec422c634feac974dee40b29bea624a5df7ea8"}
Jul 09 03:47:35.774319 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:35.774281    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr" event={"ID":"62a1507d-4169-4cc1-a312-088304e18bac","Type":"ContainerStarted","Data":"832ccd1ea3c2418a33e798ebe5c70826d56ce35f57362844002b1f66aedcc0db"}
Jul 09 03:47:35.774750 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:35.774391    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:35.810014 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:35.809952    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr" podStartSLOduration=2.226108746 podStartE2EDuration="3.809937718s" totalImagesPullingTime="1.583828972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:47:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:47:35.808939275 +0000 UTC m=+272.628195053" watchObservedRunningTime="2026-07-09 03:47:35.809937718 +0000 UTC m=+272.629193496"
Jul 09 03:47:36.111100 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:36.111062    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:36.205228 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:36.205195    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:36.719984 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:36.719949    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:36.923241 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:36.923202    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:37.020690 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.020593    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:37.070146 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.070114    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"]
Jul 09 03:47:37.072226 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.072209    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.075407 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.075387    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 03:47:37.075803 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.075783    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 03:47:37.075859 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.075809    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gpbc5\""
Jul 09 03:47:37.075920 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.075857    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 03:47:37.076409 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.076393    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 03:47:37.090461 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.090426    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"]
Jul 09 03:47:37.190554 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.190520    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b175cce-af26-480d-aa42-cac582954955-manifests\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.190554 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.190563    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b175cce-af26-480d-aa42-cac582954955-webhook-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.190816 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.190648    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b175cce-af26-480d-aa42-cac582954955-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.190816 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.190691    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbzdn\" (UniqueName: \"kubernetes.io/projected/2b175cce-af26-480d-aa42-cac582954955-kube-api-access-fbzdn\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.190816 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.190717    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b175cce-af26-480d-aa42-cac582954955-tmp\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.274208 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.274114    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:37.292102 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.292075    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b175cce-af26-480d-aa42-cac582954955-webhook-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.292227 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.292131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b175cce-af26-480d-aa42-cac582954955-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.292227 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.292167    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbzdn\" (UniqueName: \"kubernetes.io/projected/2b175cce-af26-480d-aa42-cac582954955-kube-api-access-fbzdn\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.292227 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.292188    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b175cce-af26-480d-aa42-cac582954955-tmp\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.292227 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.292226    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b175cce-af26-480d-aa42-cac582954955-manifests\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.292770 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.292741    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b175cce-af26-480d-aa42-cac582954955-manifests\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.294563 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.294534    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b175cce-af26-480d-aa42-cac582954955-tmp\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.294737 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.294716    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b175cce-af26-480d-aa42-cac582954955-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.294800 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.294779    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b175cce-af26-480d-aa42-cac582954955-webhook-cert\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.311907 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.311849    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbzdn\" (UniqueName: \"kubernetes.io/projected/2b175cce-af26-480d-aa42-cac582954955-kube-api-access-fbzdn\") pod \"opendatahub-operator-controller-manager-645c4d5897-b5kjp\" (UID: \"2b175cce-af26-480d-aa42-cac582954955\") " pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.367191 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.367157    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:37.382054 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.382023    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:37.469010 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.468971    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:37.520913 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.520889    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"]
Jul 09 03:47:37.525966 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:47:37.525880    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b175cce_af26_480d_aa42_cac582954955.slice/crio-a60277a6eec6970a4c4baceb51a8f66733a13999159ad71bea92f8912e6c39aa WatchSource:0}: Error finding container a60277a6eec6970a4c4baceb51a8f66733a13999159ad71bea92f8912e6c39aa: Status 404 returned error can't find the container with id a60277a6eec6970a4c4baceb51a8f66733a13999159ad71bea92f8912e6c39aa
Jul 09 03:47:37.568611 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.568576    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:37.669482 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.669430    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:37.770493 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.770443    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:37.784181 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:37.784153    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp" event={"ID":"2b175cce-af26-480d-aa42-cac582954955","Type":"ContainerStarted","Data":"a60277a6eec6970a4c4baceb51a8f66733a13999159ad71bea92f8912e6c39aa"}
Jul 09 03:47:40.800947 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:40.800908    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp" event={"ID":"2b175cce-af26-480d-aa42-cac582954955","Type":"ContainerStarted","Data":"b1aaae8e3973c3e64ebf4a53b87173a825094966456123d94a295cfe87bc99a3"}
Jul 09 03:47:41.807207 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:41.807169    2582 generic.go:356] "Generic (PLEG): container finished" podID="2b175cce-af26-480d-aa42-cac582954955" containerID="b1aaae8e3973c3e64ebf4a53b87173a825094966456123d94a295cfe87bc99a3" exitCode=0
Jul 09 03:47:41.807654 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:41.807262    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp" event={"ID":"2b175cce-af26-480d-aa42-cac582954955","Type":"ContainerDied","Data":"b1aaae8e3973c3e64ebf4a53b87173a825094966456123d94a295cfe87bc99a3"}
Jul 09 03:47:42.814269 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:42.814229    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp" event={"ID":"2b175cce-af26-480d-aa42-cac582954955","Type":"ContainerStarted","Data":"0f0409eee8e4ca59f9b290ee383676850924bc62f86a42095eb26e69e3768da5"}
Jul 09 03:47:42.814717 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:42.814312    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:42.841041 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:42.840989    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp" podStartSLOduration=1.318043661 podStartE2EDuration="5.840972883s" totalImagesPullingTime="3.505048213s" totalInitContainerRuntime="1.017881009s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:47:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:47:42.838617444 +0000 UTC m=+279.657873222" watchObservedRunningTime="2026-07-09 03:47:42.840972883 +0000 UTC m=+279.660228660"
Jul 09 03:47:46.781146 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:46.781117    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7f5f9c4b6d-x4tgr"
Jul 09 03:47:53.821258 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:53.821226    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-645c4d5897-b5kjp"
Jul 09 03:47:55.942159 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:55.942105    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:47:56.058560 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:47:56.058519    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:01.543245 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:01.543195    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:01.635127 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:01.635092    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:01.838139 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:01.838028    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:01.903320 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:01.903286    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:02.004311 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:02.004256    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:02.153261 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:02.153160    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:02.251358 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:02.251310    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:02.491405 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:02.491307    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:02.695462 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:02.695425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:02.818430 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:02.818380    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:02.886471 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:02.886438    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:03.547781 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:03.547753    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 03:48:03.551369 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:03.551000    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 03:48:03.559881 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:03.559841    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 03:48:03.560830 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:03.560809    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 03:48:03.563647 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:03.563624    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 03:48:03.684482 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:03.684295    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:03.918700 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:03.918607    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:04.019816 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:04.019782    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:08.224843 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.224804    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:08.769133 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.769097    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"]
Jul 09 03:48:08.773856 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.773832    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.776633 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.776612    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 09 03:48:08.776633 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.776622    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 09 03:48:08.776831 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.776675    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 03:48:08.776831 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.776706    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 09 03:48:08.776831 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.776710    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-n6b5h\""
Jul 09 03:48:08.776831 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.776784    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 09 03:48:08.776831 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.776790    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 03:48:08.784456 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.784435    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"]
Jul 09 03:48:08.838810 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.838766    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.839025 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.838894    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b285065-affa-4582-9997-e5ac1a4ac3e9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.839025 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.838932    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.839025 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.838974    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.839189 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.839103    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.839189 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.839166    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7b285065-affa-4582-9997-e5ac1a4ac3e9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.839290 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.839199    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fppzn\" (UniqueName: \"kubernetes.io/projected/7b285065-affa-4582-9997-e5ac1a4ac3e9-kube-api-access-fppzn\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.914337 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.914297    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:08.939766 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.939737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.939766 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.939771    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b285065-affa-4582-9997-e5ac1a4ac3e9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.940062 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.939795    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.940062 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.939831    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.940062 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.939852    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.940062 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.939904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7b285065-affa-4582-9997-e5ac1a4ac3e9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.940062 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.939929    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fppzn\" (UniqueName: \"kubernetes.io/projected/7b285065-affa-4582-9997-e5ac1a4ac3e9-kube-api-access-fppzn\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.940719 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.940693    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.942341 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.942316    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7b285065-affa-4582-9997-e5ac1a4ac3e9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.942454 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.942381    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.942578 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.942553    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b285065-affa-4582-9997-e5ac1a4ac3e9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.942645 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.942630    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.947971 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.947945    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fppzn\" (UniqueName: \"kubernetes.io/projected/7b285065-affa-4582-9997-e5ac1a4ac3e9-kube-api-access-fppzn\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:08.948070 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:08.947970    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b285065-affa-4582-9997-e5ac1a4ac3e9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-59nvr\" (UID: \"7b285065-affa-4582-9997-e5ac1a4ac3e9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:09.083601 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:09.083557    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:09.224384 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:09.224335    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:09.250282 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:09.250249    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"]
Jul 09 03:48:09.253841 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:48:09.253810    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b285065_affa_4582_9997_e5ac1a4ac3e9.slice/crio-19d1495f7bd6a8eb5dcd22b629192475b4ea4682664697db7c4749c75bb091f9 WatchSource:0}: Error finding container 19d1495f7bd6a8eb5dcd22b629192475b4ea4682664697db7c4749c75bb091f9: Status 404 returned error can't find the container with id 19d1495f7bd6a8eb5dcd22b629192475b4ea4682664697db7c4749c75bb091f9
Jul 09 03:48:09.256189 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:09.256172    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 03:48:09.323501 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:09.323461    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:09.941856 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:09.941814    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr" event={"ID":"7b285065-affa-4582-9997-e5ac1a4ac3e9","Type":"ContainerStarted","Data":"19d1495f7bd6a8eb5dcd22b629192475b4ea4682664697db7c4749c75bb091f9"}
Jul 09 03:48:11.438321 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:11.438282    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:11.557122 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:11.557047    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:11.941500 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:11.941460    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:12.041402 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:12.041367    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:12.157835 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:12.157796    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 03:48:12.157963 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:12.157880    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 03:48:12.735777 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:12.735722    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:12.798151 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:12.798098    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:12.874084 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:12.874051    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:12.956455 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:12.956424    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr" event={"ID":"7b285065-affa-4582-9997-e5ac1a4ac3e9","Type":"ContainerStarted","Data":"8663e55eec006ab2be3db99d9b4c036a7f79fc3c4403890a5aeea6b936793a76"}
Jul 09 03:48:12.956634 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:12.956486    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:13.000836 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:13.000706    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr" podStartSLOduration=2.099474998 podStartE2EDuration="5.000686315s" totalImagesPullingTime="2.901211317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:48:12.998194068 +0000 UTC m=+309.817449846" watchObservedRunningTime="2026-07-09 03:48:13.000686315 +0000 UTC m=+309.819942094"
Jul 09 03:48:13.962654 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:13.962615    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59nvr"
Jul 09 03:48:47.987684 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:47.987644    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:48.041267 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:48.041229    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:57.314414 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:57.314327    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:57.414328 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:57.414293    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:57.555591 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:57.555553    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:58.239767 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:58.239732    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:48:58.440368 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:48:58.440333    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:18.053748 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.053710    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"]
Jul 09 03:49:18.057051 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.057031    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:18.059982 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.059959    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 03:49:18.059982 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.059978    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 03:49:18.060936 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.060922    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gkqxk\""
Jul 09 03:49:18.064947 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.064924    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"]
Jul 09 03:49:18.158807 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.158774    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"]
Jul 09 03:49:18.161955 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.161938    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:18.169786 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.169763    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"]
Jul 09 03:49:18.183492 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.183459    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnbsb\" (UniqueName: \"kubernetes.io/projected/d2c8bfbc-6e27-4051-b42d-c9679264a73d-kube-api-access-fnbsb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:18.183697 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.183514    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:18.183697 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.183626    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:18.259083 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.259046    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"]
Jul 09 03:49:18.262375 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.262357    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:18.269979 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.269955    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"]
Jul 09 03:49:18.284534 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.284504    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnbsb\" (UniqueName: \"kubernetes.io/projected/d2c8bfbc-6e27-4051-b42d-c9679264a73d-kube-api-access-fnbsb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:18.284742 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.284552    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:18.284812 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.284756    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:18.284918 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.284825    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:18.284918 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.284857    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:18.284918 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.284910    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbd6l\" (UniqueName: \"kubernetes.io/projected/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-kube-api-access-kbd6l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:18.285057 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.285007    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:18.285095 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.285072    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:18.292813 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.292784    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnbsb\" (UniqueName: \"kubernetes.io/projected/d2c8bfbc-6e27-4051-b42d-c9679264a73d-kube-api-access-fnbsb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:18.358716 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.358615    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"]
Jul 09 03:49:18.362026 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.362009    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:18.367346 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.366987    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:18.371303 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.371276    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"]
Jul 09 03:49:18.386247 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.386211    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6pc8\" (UniqueName: \"kubernetes.io/projected/e5894dae-05db-4a52-b5e4-d66e7e98b300-kube-api-access-m6pc8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:18.386408 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.386273    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:18.386408 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.386303    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:18.386408 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.386347    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:18.386408 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.386377    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbd6l\" (UniqueName: \"kubernetes.io/projected/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-kube-api-access-kbd6l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:18.386561 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.386470    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:18.386677 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.386659    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:18.386788 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.386767    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:18.394903 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.394615    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbd6l\" (UniqueName: \"kubernetes.io/projected/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-kube-api-access-kbd6l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:18.472216 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.472185    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:18.487776 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.487738    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:18.487970 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.487804    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:18.487970 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.487883    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6pc8\" (UniqueName: \"kubernetes.io/projected/e5894dae-05db-4a52-b5e4-d66e7e98b300-kube-api-access-m6pc8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:18.487970 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.487918    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2gjz\" (UniqueName: \"kubernetes.io/projected/f2d469b7-157d-42c5-a735-187c703c2f4e-kube-api-access-p2gjz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:18.487970 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.487967    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:18.488172 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.488049    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:18.488212 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.488182    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:18.488324 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.488306    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:18.497757 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.497724    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6pc8\" (UniqueName: \"kubernetes.io/projected/e5894dae-05db-4a52-b5e4-d66e7e98b300-kube-api-access-m6pc8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:18.505598 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.505574    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"]
Jul 09 03:49:18.507997 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:49:18.507969    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2c8bfbc_6e27_4051_b42d_c9679264a73d.slice/crio-7f49e6d1dd3186178924e6925ccc2285517cf78dc6de9375444536190a7cb95e WatchSource:0}: Error finding container 7f49e6d1dd3186178924e6925ccc2285517cf78dc6de9375444536190a7cb95e: Status 404 returned error can't find the container with id 7f49e6d1dd3186178924e6925ccc2285517cf78dc6de9375444536190a7cb95e
Jul 09 03:49:18.573126 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.573094    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:18.589450 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.589415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:18.589623 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.589503    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2gjz\" (UniqueName: \"kubernetes.io/projected/f2d469b7-157d-42c5-a735-187c703c2f4e-kube-api-access-p2gjz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:18.589623 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.589545    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:18.590039 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.589814    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:18.590152 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.590082    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:18.598285 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.598252    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2gjz\" (UniqueName: \"kubernetes.io/projected/f2d469b7-157d-42c5-a735-187c703c2f4e-kube-api-access-p2gjz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:18.609984 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.609951    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"]
Jul 09 03:49:18.611360 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:49:18.611329    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c2dbea_9078_4ff6_afde_9c8b46b1dd20.slice/crio-75de24f6ca5eca55ba781124a9171f775f02008a682ff87e009f22f437e25097 WatchSource:0}: Error finding container 75de24f6ca5eca55ba781124a9171f775f02008a682ff87e009f22f437e25097: Status 404 returned error can't find the container with id 75de24f6ca5eca55ba781124a9171f775f02008a682ff87e009f22f437e25097
Jul 09 03:49:18.673211 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.673171    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:18.718801 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.718771    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"]
Jul 09 03:49:18.720414 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:49:18.720383    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5894dae_05db_4a52_b5e4_d66e7e98b300.slice/crio-161de229dbc85de1562bb860679450d715e62b530212abcda9f674dcdca17e31 WatchSource:0}: Error finding container 161de229dbc85de1562bb860679450d715e62b530212abcda9f674dcdca17e31: Status 404 returned error can't find the container with id 161de229dbc85de1562bb860679450d715e62b530212abcda9f674dcdca17e31
Jul 09 03:49:18.826222 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:18.826196    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"]
Jul 09 03:49:18.834895 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:49:18.834836    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2d469b7_157d_42c5_a735_187c703c2f4e.slice/crio-c9fd4002969fd5eabb337b0ae609689239928e7d12bfbb3a048c3b1e6d30e81d WatchSource:0}: Error finding container c9fd4002969fd5eabb337b0ae609689239928e7d12bfbb3a048c3b1e6d30e81d: Status 404 returned error can't find the container with id c9fd4002969fd5eabb337b0ae609689239928e7d12bfbb3a048c3b1e6d30e81d
Jul 09 03:49:19.244008 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.243913    2582 generic.go:356] "Generic (PLEG): container finished" podID="e5894dae-05db-4a52-b5e4-d66e7e98b300" containerID="5cc9e7986b4beb6e980beb2662f198cf75e7a1c70532a56b93a8816ae26ea73e" exitCode=0
Jul 09 03:49:19.244008 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.243995    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5" event={"ID":"e5894dae-05db-4a52-b5e4-d66e7e98b300","Type":"ContainerDied","Data":"5cc9e7986b4beb6e980beb2662f198cf75e7a1c70532a56b93a8816ae26ea73e"}
Jul 09 03:49:19.244489 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.244045    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5" event={"ID":"e5894dae-05db-4a52-b5e4-d66e7e98b300","Type":"ContainerStarted","Data":"161de229dbc85de1562bb860679450d715e62b530212abcda9f674dcdca17e31"}
Jul 09 03:49:19.245786 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.245764    2582 generic.go:356] "Generic (PLEG): container finished" podID="87c2dbea-9078-4ff6-afde-9c8b46b1dd20" containerID="04d76b400787584acebafef19eb8a9c8d61f108778172c5f93643f5f262783f6" exitCode=0
Jul 09 03:49:19.245897 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.245824    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp" event={"ID":"87c2dbea-9078-4ff6-afde-9c8b46b1dd20","Type":"ContainerDied","Data":"04d76b400787584acebafef19eb8a9c8d61f108778172c5f93643f5f262783f6"}
Jul 09 03:49:19.245897 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.245888    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp" event={"ID":"87c2dbea-9078-4ff6-afde-9c8b46b1dd20","Type":"ContainerStarted","Data":"75de24f6ca5eca55ba781124a9171f775f02008a682ff87e009f22f437e25097"}
Jul 09 03:49:19.247386 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.247365    2582 generic.go:356] "Generic (PLEG): container finished" podID="d2c8bfbc-6e27-4051-b42d-c9679264a73d" containerID="ebbb74397870c65406f83dea540be2cb7a124137eb36397390682c75a9738ea9" exitCode=0
Jul 09 03:49:19.247507 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.247450    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6" event={"ID":"d2c8bfbc-6e27-4051-b42d-c9679264a73d","Type":"ContainerDied","Data":"ebbb74397870c65406f83dea540be2cb7a124137eb36397390682c75a9738ea9"}
Jul 09 03:49:19.247507 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.247485    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6" event={"ID":"d2c8bfbc-6e27-4051-b42d-c9679264a73d","Type":"ContainerStarted","Data":"7f49e6d1dd3186178924e6925ccc2285517cf78dc6de9375444536190a7cb95e"}
Jul 09 03:49:19.249256 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.249227    2582 generic.go:356] "Generic (PLEG): container finished" podID="f2d469b7-157d-42c5-a735-187c703c2f4e" containerID="fb64963b007a18a67b4b347e4a03fcd5ce9cf3a026cee7775c4538350452f54d" exitCode=0
Jul 09 03:49:19.249359 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.249268    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf" event={"ID":"f2d469b7-157d-42c5-a735-187c703c2f4e","Type":"ContainerDied","Data":"fb64963b007a18a67b4b347e4a03fcd5ce9cf3a026cee7775c4538350452f54d"}
Jul 09 03:49:19.249359 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:19.249301    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf" event={"ID":"f2d469b7-157d-42c5-a735-187c703c2f4e","Type":"ContainerStarted","Data":"c9fd4002969fd5eabb337b0ae609689239928e7d12bfbb3a048c3b1e6d30e81d"}
Jul 09 03:49:20.255622 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:20.255579    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp" event={"ID":"87c2dbea-9078-4ff6-afde-9c8b46b1dd20","Type":"ContainerStarted","Data":"7fe713f74f4d0b22f92d8c3dff09bc66e95842a5c758a886b5e4a61f207cb0ef"}
Jul 09 03:49:20.257476 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:20.257445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf" event={"ID":"f2d469b7-157d-42c5-a735-187c703c2f4e","Type":"ContainerStarted","Data":"47dd605169fb0854ecd70e0505ea7d88ef86f0652672d419a0706b96f2a99c64"}
Jul 09 03:49:20.259183 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:20.259160    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5" event={"ID":"e5894dae-05db-4a52-b5e4-d66e7e98b300","Type":"ContainerStarted","Data":"9871bc2a7cb14ec1277852edf9872a910f17eeab7d56b7a4839704e059959255"}
Jul 09 03:49:21.266304 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:21.266266    2582 generic.go:356] "Generic (PLEG): container finished" podID="d2c8bfbc-6e27-4051-b42d-c9679264a73d" containerID="98570fbe1655972b961d710674946bb309006f59b594bb79083453a85343b4b8" exitCode=0
Jul 09 03:49:21.266772 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:21.266348    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6" event={"ID":"d2c8bfbc-6e27-4051-b42d-c9679264a73d","Type":"ContainerDied","Data":"98570fbe1655972b961d710674946bb309006f59b594bb79083453a85343b4b8"}
Jul 09 03:49:21.268369 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:21.268344    2582 generic.go:356] "Generic (PLEG): container finished" podID="f2d469b7-157d-42c5-a735-187c703c2f4e" containerID="47dd605169fb0854ecd70e0505ea7d88ef86f0652672d419a0706b96f2a99c64" exitCode=0
Jul 09 03:49:21.268450 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:21.268424    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf" event={"ID":"f2d469b7-157d-42c5-a735-187c703c2f4e","Type":"ContainerDied","Data":"47dd605169fb0854ecd70e0505ea7d88ef86f0652672d419a0706b96f2a99c64"}
Jul 09 03:49:21.270344 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:21.270323    2582 generic.go:356] "Generic (PLEG): container finished" podID="e5894dae-05db-4a52-b5e4-d66e7e98b300" containerID="9871bc2a7cb14ec1277852edf9872a910f17eeab7d56b7a4839704e059959255" exitCode=0
Jul 09 03:49:21.270458 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:21.270400    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5" event={"ID":"e5894dae-05db-4a52-b5e4-d66e7e98b300","Type":"ContainerDied","Data":"9871bc2a7cb14ec1277852edf9872a910f17eeab7d56b7a4839704e059959255"}
Jul 09 03:49:21.272439 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:21.272418    2582 generic.go:356] "Generic (PLEG): container finished" podID="87c2dbea-9078-4ff6-afde-9c8b46b1dd20" containerID="7fe713f74f4d0b22f92d8c3dff09bc66e95842a5c758a886b5e4a61f207cb0ef" exitCode=0
Jul 09 03:49:21.272516 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:21.272460    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp" event={"ID":"87c2dbea-9078-4ff6-afde-9c8b46b1dd20","Type":"ContainerDied","Data":"7fe713f74f4d0b22f92d8c3dff09bc66e95842a5c758a886b5e4a61f207cb0ef"}
Jul 09 03:49:22.278674 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:22.278637    2582 generic.go:356] "Generic (PLEG): container finished" podID="d2c8bfbc-6e27-4051-b42d-c9679264a73d" containerID="8d16324b4ef4d11eeece3185cc29bdc2e3ff4a38a8b499af0aa27363e7367faf" exitCode=0
Jul 09 03:49:22.278674 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:22.278675    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6" event={"ID":"d2c8bfbc-6e27-4051-b42d-c9679264a73d","Type":"ContainerDied","Data":"8d16324b4ef4d11eeece3185cc29bdc2e3ff4a38a8b499af0aa27363e7367faf"}
Jul 09 03:49:22.280742 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:22.280721    2582 generic.go:356] "Generic (PLEG): container finished" podID="f2d469b7-157d-42c5-a735-187c703c2f4e" containerID="b87830ad4733cf2ec2eaec7c0fcf48efa35d9ab8cb2cb6e4b7c73586a0d47101" exitCode=0
Jul 09 03:49:22.280849 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:22.280799    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf" event={"ID":"f2d469b7-157d-42c5-a735-187c703c2f4e","Type":"ContainerDied","Data":"b87830ad4733cf2ec2eaec7c0fcf48efa35d9ab8cb2cb6e4b7c73586a0d47101"}
Jul 09 03:49:22.282962 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:22.282942    2582 generic.go:356] "Generic (PLEG): container finished" podID="e5894dae-05db-4a52-b5e4-d66e7e98b300" containerID="80f824b483b5c5832089b13a06880fb356f51d8993d7a236f34a72ab6017ec93" exitCode=0
Jul 09 03:49:22.283068 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:22.283020    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5" event={"ID":"e5894dae-05db-4a52-b5e4-d66e7e98b300","Type":"ContainerDied","Data":"80f824b483b5c5832089b13a06880fb356f51d8993d7a236f34a72ab6017ec93"}
Jul 09 03:49:22.285025 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:22.285008    2582 generic.go:356] "Generic (PLEG): container finished" podID="87c2dbea-9078-4ff6-afde-9c8b46b1dd20" containerID="b803b146ce2a58a70ffe94915f178e0742ca682cfc6d6feeb7e7b28f9c905bff" exitCode=0
Jul 09 03:49:22.285102 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:22.285045    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp" event={"ID":"87c2dbea-9078-4ff6-afde-9c8b46b1dd20","Type":"ContainerDied","Data":"b803b146ce2a58a70ffe94915f178e0742ca682cfc6d6feeb7e7b28f9c905bff"}
Jul 09 03:49:23.436393 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.436336    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:23.505999 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.505971    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:23.509522 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.509498    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:23.513063 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.513045    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:23.533626 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.533597    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-util\") pod \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") "
Jul 09 03:49:23.533804 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.533645    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnbsb\" (UniqueName: \"kubernetes.io/projected/d2c8bfbc-6e27-4051-b42d-c9679264a73d-kube-api-access-fnbsb\") pod \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") "
Jul 09 03:49:23.533804 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.533675    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-bundle\") pod \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\" (UID: \"d2c8bfbc-6e27-4051-b42d-c9679264a73d\") "
Jul 09 03:49:23.534260 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.534232    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-bundle" (OuterVolumeSpecName: "bundle") pod "d2c8bfbc-6e27-4051-b42d-c9679264a73d" (UID: "d2c8bfbc-6e27-4051-b42d-c9679264a73d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:49:23.535904 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.535859    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2c8bfbc-6e27-4051-b42d-c9679264a73d-kube-api-access-fnbsb" (OuterVolumeSpecName: "kube-api-access-fnbsb") pod "d2c8bfbc-6e27-4051-b42d-c9679264a73d" (UID: "d2c8bfbc-6e27-4051-b42d-c9679264a73d"). InnerVolumeSpecName "kube-api-access-fnbsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:49:23.539917 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.539888    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-util" (OuterVolumeSpecName: "util") pod "d2c8bfbc-6e27-4051-b42d-c9679264a73d" (UID: "d2c8bfbc-6e27-4051-b42d-c9679264a73d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:49:23.634393 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634303    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-util\") pod \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") "
Jul 09 03:49:23.634393 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634341    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-bundle\") pod \"f2d469b7-157d-42c5-a735-187c703c2f4e\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") "
Jul 09 03:49:23.634393 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634382    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-bundle\") pod \"e5894dae-05db-4a52-b5e4-d66e7e98b300\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") "
Jul 09 03:49:23.634741 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634405    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbd6l\" (UniqueName: \"kubernetes.io/projected/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-kube-api-access-kbd6l\") pod \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") "
Jul 09 03:49:23.634741 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634434    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6pc8\" (UniqueName: \"kubernetes.io/projected/e5894dae-05db-4a52-b5e4-d66e7e98b300-kube-api-access-m6pc8\") pod \"e5894dae-05db-4a52-b5e4-d66e7e98b300\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") "
Jul 09 03:49:23.634741 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634458    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2gjz\" (UniqueName: \"kubernetes.io/projected/f2d469b7-157d-42c5-a735-187c703c2f4e-kube-api-access-p2gjz\") pod \"f2d469b7-157d-42c5-a735-187c703c2f4e\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") "
Jul 09 03:49:23.634741 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634486    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-util\") pod \"f2d469b7-157d-42c5-a735-187c703c2f4e\" (UID: \"f2d469b7-157d-42c5-a735-187c703c2f4e\") "
Jul 09 03:49:23.634741 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634509    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-bundle\") pod \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\" (UID: \"87c2dbea-9078-4ff6-afde-9c8b46b1dd20\") "
Jul 09 03:49:23.634741 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634532    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-util\") pod \"e5894dae-05db-4a52-b5e4-d66e7e98b300\" (UID: \"e5894dae-05db-4a52-b5e4-d66e7e98b300\") "
Jul 09 03:49:23.634741 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634705    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-util\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:23.634741 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634726    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnbsb\" (UniqueName: \"kubernetes.io/projected/d2c8bfbc-6e27-4051-b42d-c9679264a73d-kube-api-access-fnbsb\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:23.634741 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.634740    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2c8bfbc-6e27-4051-b42d-c9679264a73d-bundle\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:23.635213 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.635184    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-bundle" (OuterVolumeSpecName: "bundle") pod "e5894dae-05db-4a52-b5e4-d66e7e98b300" (UID: "e5894dae-05db-4a52-b5e4-d66e7e98b300"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:49:23.635360 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.635294    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-bundle" (OuterVolumeSpecName: "bundle") pod "87c2dbea-9078-4ff6-afde-9c8b46b1dd20" (UID: "87c2dbea-9078-4ff6-afde-9c8b46b1dd20"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:49:23.635490 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.635366    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-bundle" (OuterVolumeSpecName: "bundle") pod "f2d469b7-157d-42c5-a735-187c703c2f4e" (UID: "f2d469b7-157d-42c5-a735-187c703c2f4e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:49:23.637058 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.637022    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-kube-api-access-kbd6l" (OuterVolumeSpecName: "kube-api-access-kbd6l") pod "87c2dbea-9078-4ff6-afde-9c8b46b1dd20" (UID: "87c2dbea-9078-4ff6-afde-9c8b46b1dd20"). InnerVolumeSpecName "kube-api-access-kbd6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:49:23.637058 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.637022    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5894dae-05db-4a52-b5e4-d66e7e98b300-kube-api-access-m6pc8" (OuterVolumeSpecName: "kube-api-access-m6pc8") pod "e5894dae-05db-4a52-b5e4-d66e7e98b300" (UID: "e5894dae-05db-4a52-b5e4-d66e7e98b300"). InnerVolumeSpecName "kube-api-access-m6pc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:49:23.637586 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.637569    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2d469b7-157d-42c5-a735-187c703c2f4e-kube-api-access-p2gjz" (OuterVolumeSpecName: "kube-api-access-p2gjz") pod "f2d469b7-157d-42c5-a735-187c703c2f4e" (UID: "f2d469b7-157d-42c5-a735-187c703c2f4e"). InnerVolumeSpecName "kube-api-access-p2gjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:49:23.639817 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.639795    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-util" (OuterVolumeSpecName: "util") pod "e5894dae-05db-4a52-b5e4-d66e7e98b300" (UID: "e5894dae-05db-4a52-b5e4-d66e7e98b300"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:49:23.640920 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.640891    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-util" (OuterVolumeSpecName: "util") pod "f2d469b7-157d-42c5-a735-187c703c2f4e" (UID: "f2d469b7-157d-42c5-a735-187c703c2f4e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:49:23.641468 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.641447    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-util" (OuterVolumeSpecName: "util") pod "87c2dbea-9078-4ff6-afde-9c8b46b1dd20" (UID: "87c2dbea-9078-4ff6-afde-9c8b46b1dd20"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:49:23.735927 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.735890    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-bundle\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:23.735927 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.735927    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbd6l\" (UniqueName: \"kubernetes.io/projected/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-kube-api-access-kbd6l\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:23.735927 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.735937    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6pc8\" (UniqueName: \"kubernetes.io/projected/e5894dae-05db-4a52-b5e4-d66e7e98b300-kube-api-access-m6pc8\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:23.736118 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.735946    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2gjz\" (UniqueName: \"kubernetes.io/projected/f2d469b7-157d-42c5-a735-187c703c2f4e-kube-api-access-p2gjz\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:23.736118 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.735956    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-util\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:23.736118 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.735964    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-bundle\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:23.736118 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.735971    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5894dae-05db-4a52-b5e4-d66e7e98b300-util\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:23.736118 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.735978    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c2dbea-9078-4ff6-afde-9c8b46b1dd20-util\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:23.736118 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:23.735987    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d469b7-157d-42c5-a735-187c703c2f4e-bundle\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:24.296764 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.296741    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5"
Jul 09 03:49:24.296764 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.296752    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5" event={"ID":"e5894dae-05db-4a52-b5e4-d66e7e98b300","Type":"ContainerDied","Data":"161de229dbc85de1562bb860679450d715e62b530212abcda9f674dcdca17e31"}
Jul 09 03:49:24.297022 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.296807    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="161de229dbc85de1562bb860679450d715e62b530212abcda9f674dcdca17e31"
Jul 09 03:49:24.298608 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.298536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp" event={"ID":"87c2dbea-9078-4ff6-afde-9c8b46b1dd20","Type":"ContainerDied","Data":"75de24f6ca5eca55ba781124a9171f775f02008a682ff87e009f22f437e25097"}
Jul 09 03:49:24.298608 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.298572    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp"
Jul 09 03:49:24.298608 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.298609    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75de24f6ca5eca55ba781124a9171f775f02008a682ff87e009f22f437e25097"
Jul 09 03:49:24.300825 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.300801    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6" event={"ID":"d2c8bfbc-6e27-4051-b42d-c9679264a73d","Type":"ContainerDied","Data":"7f49e6d1dd3186178924e6925ccc2285517cf78dc6de9375444536190a7cb95e"}
Jul 09 03:49:24.300964 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.300845    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f49e6d1dd3186178924e6925ccc2285517cf78dc6de9375444536190a7cb95e"
Jul 09 03:49:24.301046 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.300812    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6"
Jul 09 03:49:24.302904 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.302885    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf" event={"ID":"f2d469b7-157d-42c5-a735-187c703c2f4e","Type":"ContainerDied","Data":"c9fd4002969fd5eabb337b0ae609689239928e7d12bfbb3a048c3b1e6d30e81d"}
Jul 09 03:49:24.303008 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.302922    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9fd4002969fd5eabb337b0ae609689239928e7d12bfbb3a048c3b1e6d30e81d"
Jul 09 03:49:24.303008 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:24.302929    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf"
Jul 09 03:49:31.066483 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:31.066447    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:34.672419 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:34.672378    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:37.689669 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:37.689618    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:39.150801 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:39.150772    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:40.102513 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:40.102467    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:42.258117 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:42.258079    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:42.658280 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:42.658244    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:42.902576 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:42.902532    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:43.021580 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:43.021478    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:47.505304 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:47.505268    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:48.217734 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:48.217695    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:48.509014 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:48.508922    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:49.466412 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:49.466375    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:49.566297 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:49.566257    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:53.268358 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:53.268317    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:53.368765 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:53.368732    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:49:58.620214 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620169    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"]
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620460    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5894dae-05db-4a52-b5e4-d66e7e98b300" containerName="util"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620472    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5894dae-05db-4a52-b5e4-d66e7e98b300" containerName="util"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620481    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2c8bfbc-6e27-4051-b42d-c9679264a73d" containerName="pull"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620486    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2c8bfbc-6e27-4051-b42d-c9679264a73d" containerName="pull"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620495    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d469b7-157d-42c5-a735-187c703c2f4e" containerName="extract"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620503    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d469b7-157d-42c5-a735-187c703c2f4e" containerName="extract"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620519    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c2dbea-9078-4ff6-afde-9c8b46b1dd20" containerName="util"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620526    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c2dbea-9078-4ff6-afde-9c8b46b1dd20" containerName="util"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620535    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d469b7-157d-42c5-a735-187c703c2f4e" containerName="pull"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620540    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d469b7-157d-42c5-a735-187c703c2f4e" containerName="pull"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620546    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c2dbea-9078-4ff6-afde-9c8b46b1dd20" containerName="pull"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620552    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c2dbea-9078-4ff6-afde-9c8b46b1dd20" containerName="pull"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620559    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2c8bfbc-6e27-4051-b42d-c9679264a73d" containerName="util"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620564    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2c8bfbc-6e27-4051-b42d-c9679264a73d" containerName="util"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620571    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2c8bfbc-6e27-4051-b42d-c9679264a73d" containerName="extract"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620576    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2c8bfbc-6e27-4051-b42d-c9679264a73d" containerName="extract"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620582    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d469b7-157d-42c5-a735-187c703c2f4e" containerName="util"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620587    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d469b7-157d-42c5-a735-187c703c2f4e" containerName="util"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620593    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5894dae-05db-4a52-b5e4-d66e7e98b300" containerName="extract"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620597    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5894dae-05db-4a52-b5e4-d66e7e98b300" containerName="extract"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620604    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5894dae-05db-4a52-b5e4-d66e7e98b300" containerName="pull"
Jul 09 03:49:58.620603 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620610    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5894dae-05db-4a52-b5e4-d66e7e98b300" containerName="pull"
Jul 09 03:49:58.621336 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620616    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c2dbea-9078-4ff6-afde-9c8b46b1dd20" containerName="extract"
Jul 09 03:49:58.621336 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620622    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c2dbea-9078-4ff6-afde-9c8b46b1dd20" containerName="extract"
Jul 09 03:49:58.621336 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620673    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5894dae-05db-4a52-b5e4-d66e7e98b300" containerName="extract"
Jul 09 03:49:58.621336 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620680    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="87c2dbea-9078-4ff6-afde-9c8b46b1dd20" containerName="extract"
Jul 09 03:49:58.621336 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620687    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2c8bfbc-6e27-4051-b42d-c9679264a73d" containerName="extract"
Jul 09 03:49:58.621336 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.620692    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2d469b7-157d-42c5-a735-187c703c2f4e" containerName="extract"
Jul 09 03:49:58.623332 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.623314    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:49:58.626211 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.626189    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 03:49:58.626363 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.626243    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 03:49:58.626363 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.626312    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fhkmc\""
Jul 09 03:49:58.637194 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.637167    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"]
Jul 09 03:49:58.709586 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.709545    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zcm46\" (UID: \"95a2ae61-dfef-4bc8-878f-a35e6e3c61b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:49:58.709782 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.709611    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksp72\" (UniqueName: \"kubernetes.io/projected/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-kube-api-access-ksp72\") pod \"kuadrant-operator-controller-manager-55467d9f7-zcm46\" (UID: \"95a2ae61-dfef-4bc8-878f-a35e6e3c61b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:49:58.727024 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.726992    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"]
Jul 09 03:49:58.727261 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:49:58.727239    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-ksp72], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46" podUID="95a2ae61-dfef-4bc8-878f-a35e6e3c61b0"
Jul 09 03:49:58.774986 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.774949    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"]
Jul 09 03:49:58.780611 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.780584    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"]
Jul 09 03:49:58.810249 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.810213    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksp72\" (UniqueName: \"kubernetes.io/projected/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-kube-api-access-ksp72\") pod \"kuadrant-operator-controller-manager-55467d9f7-zcm46\" (UID: \"95a2ae61-dfef-4bc8-878f-a35e6e3c61b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:49:58.810444 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.810404    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zcm46\" (UID: \"95a2ae61-dfef-4bc8-878f-a35e6e3c61b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:49:58.810775 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.810755    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zcm46\" (UID: \"95a2ae61-dfef-4bc8-878f-a35e6e3c61b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:49:58.812534 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:49:58.812510    2582 projected.go:196] Error preparing data for projected volume kube-api-access-ksp72 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46: failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-134-164.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-134-164.ec2.internal' and this object
Jul 09 03:49:58.812687 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:49:58.812584    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-kube-api-access-ksp72 podName:95a2ae61-dfef-4bc8-878f-a35e6e3c61b0 nodeName:}" failed. No retries permitted until 2026-07-09 03:49:59.312567172 +0000 UTC m=+416.131822928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ksp72" (UniqueName: "kubernetes.io/projected/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-kube-api-access-ksp72") pod "kuadrant-operator-controller-manager-55467d9f7-zcm46" (UID: "95a2ae61-dfef-4bc8-878f-a35e6e3c61b0") : failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-134-164.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-134-164.ec2.internal' and this object
Jul 09 03:49:58.837768 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.837732    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"]
Jul 09 03:49:58.840832 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.840814    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:49:58.866751 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:58.866712    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"]
Jul 09 03:49:59.012688 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.012591    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4fsxz\" (UID: \"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:49:59.012688 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.012644    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv7d8\" (UniqueName: \"kubernetes.io/projected/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-kube-api-access-gv7d8\") pod \"kuadrant-operator-controller-manager-86878c769f-4fsxz\" (UID: \"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:49:59.113647 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.113607    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4fsxz\" (UID: \"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:49:59.113647 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.113646    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv7d8\" (UniqueName: \"kubernetes.io/projected/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-kube-api-access-gv7d8\") pod \"kuadrant-operator-controller-manager-86878c769f-4fsxz\" (UID: \"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:49:59.114416 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.114388    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4fsxz\" (UID: \"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:49:59.122696 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.122668    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv7d8\" (UniqueName: \"kubernetes.io/projected/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-kube-api-access-gv7d8\") pod \"kuadrant-operator-controller-manager-86878c769f-4fsxz\" (UID: \"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:49:59.152680 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.152642    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:49:59.299804 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.299777    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"]
Jul 09 03:49:59.301800 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:49:59.301764    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e7f83b_4b84_43b4_bb47_bd2a7d97b446.slice/crio-95401e0daa980578fd90d5f7af1fcf271be297c90d6fd3fc2c049fa135af9ba7 WatchSource:0}: Error finding container 95401e0daa980578fd90d5f7af1fcf271be297c90d6fd3fc2c049fa135af9ba7: Status 404 returned error can't find the container with id 95401e0daa980578fd90d5f7af1fcf271be297c90d6fd3fc2c049fa135af9ba7
Jul 09 03:49:59.316429 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.316392    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksp72\" (UniqueName: \"kubernetes.io/projected/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-kube-api-access-ksp72\") pod \"kuadrant-operator-controller-manager-55467d9f7-zcm46\" (UID: \"95a2ae61-dfef-4bc8-878f-a35e6e3c61b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:49:59.321265 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:49:59.321242    2582 projected.go:196] Error preparing data for projected volume kube-api-access-ksp72 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-zcm46" not found
Jul 09 03:49:59.321409 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:49:59.321310    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-kube-api-access-ksp72 podName:95a2ae61-dfef-4bc8-878f-a35e6e3c61b0 nodeName:}" failed. No retries permitted until 2026-07-09 03:50:00.321294108 +0000 UTC m=+417.140549867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ksp72" (UniqueName: "kubernetes.io/projected/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-kube-api-access-ksp72") pod "kuadrant-operator-controller-manager-55467d9f7-zcm46" (UID: "95a2ae61-dfef-4bc8-878f-a35e6e3c61b0") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-zcm46" not found
Jul 09 03:49:59.468440 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.468404    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz" event={"ID":"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446","Type":"ContainerStarted","Data":"95401e0daa980578fd90d5f7af1fcf271be297c90d6fd3fc2c049fa135af9ba7"}
Jul 09 03:49:59.468440 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.468441    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:49:59.470826 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:49:59.470794    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-zcm46\" is forbidden: User \"system:node:ip-10-0-134-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-164.ec2.internal' and this object" podUID="95a2ae61-dfef-4bc8-878f-a35e6e3c61b0" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:49:59.476476 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.476451    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:49:59.478569 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:49:59.478542    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-zcm46\" is forbidden: User \"system:node:ip-10-0-134-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-164.ec2.internal' and this object" podUID="95a2ae61-dfef-4bc8-878f-a35e6e3c61b0" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:49:59.619450 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.619339    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-extensions-socket-volume\") pod \"95a2ae61-dfef-4bc8-878f-a35e6e3c61b0\" (UID: \"95a2ae61-dfef-4bc8-878f-a35e6e3c61b0\") "
Jul 09 03:49:59.619600 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.619553    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksp72\" (UniqueName: \"kubernetes.io/projected/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-kube-api-access-ksp72\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:49:59.619661 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.619628    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "95a2ae61-dfef-4bc8-878f-a35e6e3c61b0" (UID: "95a2ae61-dfef-4bc8-878f-a35e6e3c61b0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:49:59.713392 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.713362    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95a2ae61-dfef-4bc8-878f-a35e6e3c61b0" path="/var/lib/kubelet/pods/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0/volumes"
Jul 09 03:49:59.720582 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:49:59.720560    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95a2ae61-dfef-4bc8-878f-a35e6e3c61b0-extensions-socket-volume\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:50:00.475495 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:00.475462    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:50:00.480965 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:50:00.480932    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-zcm46\" is forbidden: User \"system:node:ip-10-0-134-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-164.ec2.internal' and this object" podUID="95a2ae61-dfef-4bc8-878f-a35e6e3c61b0" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zcm46"
Jul 09 03:50:00.522965 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:00.522926    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:00.688475 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:00.688419    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:04.500821 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:04.500788    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz" event={"ID":"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446","Type":"ContainerStarted","Data":"86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2"}
Jul 09 03:50:04.501268 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:04.500908    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:50:04.522502 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:04.522442    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz" podStartSLOduration=1.485992994 podStartE2EDuration="6.522423133s" totalImagesPullingTime="5.036430139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:50:04.519391049 +0000 UTC m=+421.338646827" watchObservedRunningTime="2026-07-09 03:50:04.522423133 +0000 UTC m=+421.341678911"
Jul 09 03:50:05.297310 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:05.297273    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:05.395804 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:05.395770    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:07.771105 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:07.771069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:07.872982 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:07.872933    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:08.979097 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:08.979060    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:09.038984 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:09.038946    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:11.128660 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:11.128617    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:11.227933 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:11.227900    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:15.508327 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:15.508292    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:50:18.160740 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:18.160701    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:18.261993 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:18.261951    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:19.723764 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:19.723728    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:22.977985 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:22.977885    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:23.032499 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:23.032459    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:25.557032 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:25.556991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:25.658748 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:25.658706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:29.154517 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:29.154486    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:29.254954 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:29.254917    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:31.931716 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:31.931678    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:32.047524 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:32.047489    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:34.226031 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:34.225992    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:34.343985 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:34.343951    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:34.394665 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:34.394622    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:35.025271 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.025232    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-9dnt4"]
Jul 09 03:50:35.030569 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.030546    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9dnt4"
Jul 09 03:50:35.034456 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.034182    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-q95d2\""
Jul 09 03:50:35.039892 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.037404    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-9dnt4"]
Jul 09 03:50:35.052045 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.052013    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:35.106097 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.106063    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4thks\" (UniqueName: \"kubernetes.io/projected/77311a36-1a8c-45b9-bc09-e4e5e11e525b-kube-api-access-4thks\") pod \"authorino-58cd955675-9dnt4\" (UID: \"77311a36-1a8c-45b9-bc09-e4e5e11e525b\") " pod="kuadrant-system/authorino-58cd955675-9dnt4"
Jul 09 03:50:35.109530 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.109504    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:35.207568 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.207533    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4thks\" (UniqueName: \"kubernetes.io/projected/77311a36-1a8c-45b9-bc09-e4e5e11e525b-kube-api-access-4thks\") pod \"authorino-58cd955675-9dnt4\" (UID: \"77311a36-1a8c-45b9-bc09-e4e5e11e525b\") " pod="kuadrant-system/authorino-58cd955675-9dnt4"
Jul 09 03:50:35.217229 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.217195    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4thks\" (UniqueName: \"kubernetes.io/projected/77311a36-1a8c-45b9-bc09-e4e5e11e525b-kube-api-access-4thks\") pod \"authorino-58cd955675-9dnt4\" (UID: \"77311a36-1a8c-45b9-bc09-e4e5e11e525b\") " pod="kuadrant-system/authorino-58cd955675-9dnt4"
Jul 09 03:50:35.347235 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.347194    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9dnt4"
Jul 09 03:50:35.481490 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.481463    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-9dnt4"]
Jul 09 03:50:35.483245 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:50:35.483218    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77311a36_1a8c_45b9_bc09_e4e5e11e525b.slice/crio-bb87e1c6d22aa94973951258e69ccbce2668449c609e62cf428fc29bf4a85b81 WatchSource:0}: Error finding container bb87e1c6d22aa94973951258e69ccbce2668449c609e62cf428fc29bf4a85b81: Status 404 returned error can't find the container with id bb87e1c6d22aa94973951258e69ccbce2668449c609e62cf428fc29bf4a85b81
Jul 09 03:50:35.647967 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:35.647883    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9dnt4" event={"ID":"77311a36-1a8c-45b9-bc09-e4e5e11e525b","Type":"ContainerStarted","Data":"bb87e1c6d22aa94973951258e69ccbce2668449c609e62cf428fc29bf4a85b81"}
Jul 09 03:50:36.968635 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:36.968601    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:37.066777 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:37.066742    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:39.670788 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:39.670747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9dnt4" event={"ID":"77311a36-1a8c-45b9-bc09-e4e5e11e525b","Type":"ContainerStarted","Data":"9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7"}
Jul 09 03:50:39.684774 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:39.684713    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-9dnt4" podStartSLOduration=1.4288933799999999 podStartE2EDuration="4.684691676s" totalImagesPullingTime="3.255798296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:50:39.684000411 +0000 UTC m=+456.503256182" watchObservedRunningTime="2026-07-09 03:50:39.684691676 +0000 UTC m=+456.503947455"
Jul 09 03:50:39.959831 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:39.959746    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-9dnt4"]
Jul 09 03:50:41.679633 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:41.679591    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-9dnt4" podUID="77311a36-1a8c-45b9-bc09-e4e5e11e525b" containerName="authorino" containerID="cri-o://9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7" gracePeriod=30
Jul 09 03:50:41.916883 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:41.916840    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9dnt4"
Jul 09 03:50:41.965791 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:41.965695    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4thks\" (UniqueName: \"kubernetes.io/projected/77311a36-1a8c-45b9-bc09-e4e5e11e525b-kube-api-access-4thks\") pod \"77311a36-1a8c-45b9-bc09-e4e5e11e525b\" (UID: \"77311a36-1a8c-45b9-bc09-e4e5e11e525b\") "
Jul 09 03:50:41.967779 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:41.967747    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77311a36-1a8c-45b9-bc09-e4e5e11e525b-kube-api-access-4thks" (OuterVolumeSpecName: "kube-api-access-4thks") pod "77311a36-1a8c-45b9-bc09-e4e5e11e525b" (UID: "77311a36-1a8c-45b9-bc09-e4e5e11e525b"). InnerVolumeSpecName "kube-api-access-4thks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:50:42.066523 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.066467    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4thks\" (UniqueName: \"kubernetes.io/projected/77311a36-1a8c-45b9-bc09-e4e5e11e525b-kube-api-access-4thks\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:50:42.154045 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.154007    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:42.207763 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.207725    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:42.349096 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.349052    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:42.472930 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.472892    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:42.685561 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.685474    2582 generic.go:356] "Generic (PLEG): container finished" podID="77311a36-1a8c-45b9-bc09-e4e5e11e525b" containerID="9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7" exitCode=0
Jul 09 03:50:42.685561 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.685514    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9dnt4"
Jul 09 03:50:42.686063 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.685564    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9dnt4" event={"ID":"77311a36-1a8c-45b9-bc09-e4e5e11e525b","Type":"ContainerDied","Data":"9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7"}
Jul 09 03:50:42.686063 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.685607    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9dnt4" event={"ID":"77311a36-1a8c-45b9-bc09-e4e5e11e525b","Type":"ContainerDied","Data":"bb87e1c6d22aa94973951258e69ccbce2668449c609e62cf428fc29bf4a85b81"}
Jul 09 03:50:42.686063 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.685630    2582 scope.go:117] "RemoveContainer" containerID="9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7"
Jul 09 03:50:42.694521 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.694503    2582 scope.go:117] "RemoveContainer" containerID="9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7"
Jul 09 03:50:42.694798 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:50:42.694775    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7\": container with ID starting with 9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7 not found: ID does not exist" containerID="9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7"
Jul 09 03:50:42.694919 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.694804    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7"} err="failed to get container status \"9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7\": rpc error: code = NotFound desc = could not find container \"9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7\": container with ID starting with 9202c2dae9c33bb838ae8a6e223b52c8efa5413f5ee2b6f2697a45c43511f4b7 not found: ID does not exist"
Jul 09 03:50:42.708953 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.708922    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-9dnt4"]
Jul 09 03:50:42.712468 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:42.712445    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-9dnt4"]
Jul 09 03:50:43.713779 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:43.713748    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77311a36-1a8c-45b9-bc09-e4e5e11e525b" path="/var/lib/kubelet/pods/77311a36-1a8c-45b9-bc09-e4e5e11e525b/volumes"
Jul 09 03:50:50.331528 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:50.331488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:50.408266 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:50.408230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:50.801997 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:50.801956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:54.406583 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:54.406546    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:50:54.456945 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:50:54.456826    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:02.765191 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:02.765156    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:02.989116 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:02.989082    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fx8mb"]
Jul 09 03:51:02.989364 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:02.989352    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77311a36-1a8c-45b9-bc09-e4e5e11e525b" containerName="authorino"
Jul 09 03:51:02.989409 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:02.989366    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="77311a36-1a8c-45b9-bc09-e4e5e11e525b" containerName="authorino"
Jul 09 03:51:02.989443 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:02.989423    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="77311a36-1a8c-45b9-bc09-e4e5e11e525b" containerName="authorino"
Jul 09 03:51:02.992335 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:02.992313    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fx8mb"
Jul 09 03:51:02.994859 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:02.994838    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-q95d2\""
Jul 09 03:51:02.997604 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:02.997580    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fx8mb"]
Jul 09 03:51:03.138396 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.138363    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k6nl\" (UniqueName: \"kubernetes.io/projected/73e8da73-4d67-472b-bb28-178782e5815b-kube-api-access-2k6nl\") pod \"authorino-5d4b67c446-fx8mb\" (UID: \"73e8da73-4d67-472b-bb28-178782e5815b\") " pod="kuadrant-system/authorino-5d4b67c446-fx8mb"
Jul 09 03:51:03.224847 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.224804    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fx8mb"]
Jul 09 03:51:03.225094 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:51:03.225074    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2k6nl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-fx8mb" podUID="73e8da73-4d67-472b-bb28-178782e5815b"
Jul 09 03:51:03.238743 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.238711    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k6nl\" (UniqueName: \"kubernetes.io/projected/73e8da73-4d67-472b-bb28-178782e5815b-kube-api-access-2k6nl\") pod \"authorino-5d4b67c446-fx8mb\" (UID: \"73e8da73-4d67-472b-bb28-178782e5815b\") " pod="kuadrant-system/authorino-5d4b67c446-fx8mb"
Jul 09 03:51:03.250184 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.250156    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k6nl\" (UniqueName: \"kubernetes.io/projected/73e8da73-4d67-472b-bb28-178782e5815b-kube-api-access-2k6nl\") pod \"authorino-5d4b67c446-fx8mb\" (UID: \"73e8da73-4d67-472b-bb28-178782e5815b\") " pod="kuadrant-system/authorino-5d4b67c446-fx8mb"
Jul 09 03:51:03.277889 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.277840    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:03.462964 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.462882    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5557679ddc-pss97"]
Jul 09 03:51:03.466201 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.466174    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5557679ddc-pss97"
Jul 09 03:51:03.468650 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.468626    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 09 03:51:03.472448 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.472417    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5557679ddc-pss97"]
Jul 09 03:51:03.541364 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.541330    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8mck\" (UniqueName: \"kubernetes.io/projected/d40caca6-8071-4a14-98cc-d93f17628ccc-kube-api-access-r8mck\") pod \"authorino-5557679ddc-pss97\" (UID: \"d40caca6-8071-4a14-98cc-d93f17628ccc\") " pod="kuadrant-system/authorino-5557679ddc-pss97"
Jul 09 03:51:03.541555 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.541392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d40caca6-8071-4a14-98cc-d93f17628ccc-tls-cert\") pod \"authorino-5557679ddc-pss97\" (UID: \"d40caca6-8071-4a14-98cc-d93f17628ccc\") " pod="kuadrant-system/authorino-5557679ddc-pss97"
Jul 09 03:51:03.642599 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.642564    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mck\" (UniqueName: \"kubernetes.io/projected/d40caca6-8071-4a14-98cc-d93f17628ccc-kube-api-access-r8mck\") pod \"authorino-5557679ddc-pss97\" (UID: \"d40caca6-8071-4a14-98cc-d93f17628ccc\") " pod="kuadrant-system/authorino-5557679ddc-pss97"
Jul 09 03:51:03.642797 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.642613    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d40caca6-8071-4a14-98cc-d93f17628ccc-tls-cert\") pod \"authorino-5557679ddc-pss97\" (UID: \"d40caca6-8071-4a14-98cc-d93f17628ccc\") " pod="kuadrant-system/authorino-5557679ddc-pss97"
Jul 09 03:51:03.645358 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.645335    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 09 03:51:03.650625 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.650601    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8mck\" (UniqueName: \"kubernetes.io/projected/d40caca6-8071-4a14-98cc-d93f17628ccc-kube-api-access-r8mck\") pod \"authorino-5557679ddc-pss97\" (UID: \"d40caca6-8071-4a14-98cc-d93f17628ccc\") " pod="kuadrant-system/authorino-5557679ddc-pss97"
Jul 09 03:51:03.655384 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.655358    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d40caca6-8071-4a14-98cc-d93f17628ccc-tls-cert\") pod \"authorino-5557679ddc-pss97\" (UID: \"d40caca6-8071-4a14-98cc-d93f17628ccc\") " pod="kuadrant-system/authorino-5557679ddc-pss97"
Jul 09 03:51:03.778794 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.778720    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-q95d2\""
Jul 09 03:51:03.780185 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.780166    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fx8mb"
Jul 09 03:51:03.786788 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.786765    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5557679ddc-pss97"
Jul 09 03:51:03.787754 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.787734    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fx8mb"
Jul 09 03:51:03.920956 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.915859    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5557679ddc-pss97"]
Jul 09 03:51:03.944651 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.944620    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2k6nl\" (UniqueName: \"kubernetes.io/projected/73e8da73-4d67-472b-bb28-178782e5815b-kube-api-access-2k6nl\") pod \"73e8da73-4d67-472b-bb28-178782e5815b\" (UID: \"73e8da73-4d67-472b-bb28-178782e5815b\") "
Jul 09 03:51:03.946878 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:03.946830    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73e8da73-4d67-472b-bb28-178782e5815b-kube-api-access-2k6nl" (OuterVolumeSpecName: "kube-api-access-2k6nl") pod "73e8da73-4d67-472b-bb28-178782e5815b" (UID: "73e8da73-4d67-472b-bb28-178782e5815b"). InnerVolumeSpecName "kube-api-access-2k6nl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:51:04.045711 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:04.045671    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k6nl\" (UniqueName: \"kubernetes.io/projected/73e8da73-4d67-472b-bb28-178782e5815b-kube-api-access-2k6nl\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:51:04.786401 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:04.786371    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fx8mb"
Jul 09 03:51:04.786401 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:04.786379    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5557679ddc-pss97" event={"ID":"d40caca6-8071-4a14-98cc-d93f17628ccc","Type":"ContainerStarted","Data":"71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3"}
Jul 09 03:51:04.786972 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:04.786420    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5557679ddc-pss97" event={"ID":"d40caca6-8071-4a14-98cc-d93f17628ccc","Type":"ContainerStarted","Data":"c3483c5ed209154e179ce8feda8b5131792ef250da4eafa03838937c5b0275fa"}
Jul 09 03:51:04.805663 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:04.805607    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5557679ddc-pss97" podStartSLOduration=1.45945102 podStartE2EDuration="1.805589915s" totalImagesPullingTime="346.138895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:51:04.803813331 +0000 UTC m=+481.623069108" watchObservedRunningTime="2026-07-09 03:51:04.805589915 +0000 UTC m=+481.624845691"
Jul 09 03:51:04.826186 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:04.826145    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fx8mb"]
Jul 09 03:51:04.833312 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:04.833281    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fx8mb"]
Jul 09 03:51:05.713807 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:05.713774    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73e8da73-4d67-472b-bb28-178782e5815b" path="/var/lib/kubelet/pods/73e8da73-4d67-472b-bb28-178782e5815b/volumes"
Jul 09 03:51:07.294967 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:07.294933    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:07.395159 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:07.395129    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:09.010185 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:09.010152    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:09.067275 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:09.067239    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:09.128708 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:09.124694    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:09.446271 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:09.446232    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:13.061023 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:13.060973    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:13.158386 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:13.158343    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:44.760965 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:44.760921    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:45.265355 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:45.265292    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:45.669222 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:45.669169    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:45.771719 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:45.771676    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:56.141242 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:56.141130    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:51:56.240357 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:51:56.240315    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:52:05.918517 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:05.918476    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"]
Jul 09 03:52:05.922021 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:05.921999    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:05.925910 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:05.925881    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 03:52:05.925910 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:05.925888    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 09 03:52:05.926115 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:05.925928    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 03:52:05.926115 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:05.925898    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-27f2j\""
Jul 09 03:52:05.930907 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:05.930884    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"]
Jul 09 03:52:05.936068 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:05.936034    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:52:06.018070 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.018035    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.018250 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.018090    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.018250 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.018108    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.018250 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.018173    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwmw7\" (UniqueName: \"kubernetes.io/projected/cfb87f88-2238-472e-9a92-1ac5f1f70e57-kube-api-access-mwmw7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.018250 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.018203    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb87f88-2238-472e-9a92-1ac5f1f70e57-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.018250 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.018225    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.119510 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.119472    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.119510 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.119509    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.119780 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.119538    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwmw7\" (UniqueName: \"kubernetes.io/projected/cfb87f88-2238-472e-9a92-1ac5f1f70e57-kube-api-access-mwmw7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.119780 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.119565    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb87f88-2238-472e-9a92-1ac5f1f70e57-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.119780 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.119597    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.119780 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.119647    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.120028 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.119959    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.120096 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.120078    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.120157 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.120132    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.121922 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.121899    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb87f88-2238-472e-9a92-1ac5f1f70e57-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.122167 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.122153    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb87f88-2238-472e-9a92-1ac5f1f70e57-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.127532 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.127506    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwmw7\" (UniqueName: \"kubernetes.io/projected/cfb87f88-2238-472e-9a92-1ac5f1f70e57-kube-api-access-mwmw7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr\" (UID: \"cfb87f88-2238-472e-9a92-1ac5f1f70e57\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.233159 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.232971    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:06.369019 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:06.368844    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"]
Jul 09 03:52:06.371441 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:52:06.371414    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfb87f88_2238_472e_9a92_1ac5f1f70e57.slice/crio-5dc35aa52cd22318ba012d0fd84fd49970c4d6ccd218b14764f2eae658815a92 WatchSource:0}: Error finding container 5dc35aa52cd22318ba012d0fd84fd49970c4d6ccd218b14764f2eae658815a92: Status 404 returned error can't find the container with id 5dc35aa52cd22318ba012d0fd84fd49970c4d6ccd218b14764f2eae658815a92
Jul 09 03:52:07.070374 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:07.070337    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr" event={"ID":"cfb87f88-2238-472e-9a92-1ac5f1f70e57","Type":"ContainerStarted","Data":"5dc35aa52cd22318ba012d0fd84fd49970c4d6ccd218b14764f2eae658815a92"}
Jul 09 03:52:08.076226 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:08.076190    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr" event={"ID":"cfb87f88-2238-472e-9a92-1ac5f1f70e57","Type":"ContainerStarted","Data":"a6c96857e044be2816e9f88e620e141feb16f846bec9cdb8e3fac2754d902874"}
Jul 09 03:52:08.095750 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:08.095598    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr" podStartSLOduration=1.908241822 podStartE2EDuration="3.095575786s" totalImagesPullingTime="1.187333964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:52:08.095445969 +0000 UTC m=+544.914701760" watchObservedRunningTime="2026-07-09 03:52:08.095575786 +0000 UTC m=+544.914831563"
Jul 09 03:52:13.895230 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:13.895194    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:52:16.234193 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:16.234158    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:16.234678 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:16.234221    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:16.251684 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:16.251646    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:17.132728 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:17.132697    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr"
Jul 09 03:52:19.650448 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:19.650399    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:52:34.110418 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.110381    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"]
Jul 09 03:52:34.114057 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.114034    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.116509 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.116479    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 09 03:52:34.124702 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.124674    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"]
Jul 09 03:52:34.158050 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.158012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.158221 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.158059    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62b89f9f-c9e9-420f-9cec-c33094408036-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.158221 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.158131    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.158221 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.158169    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9g6d\" (UniqueName: \"kubernetes.io/projected/62b89f9f-c9e9-420f-9cec-c33094408036-kube-api-access-s9g6d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.158221 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.158205    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.158355 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.158238    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.221020 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.220957    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:52:34.258783 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.258753    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9g6d\" (UniqueName: \"kubernetes.io/projected/62b89f9f-c9e9-420f-9cec-c33094408036-kube-api-access-s9g6d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.259004 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.258801    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.259004 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.258824    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.259004 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.258858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.259004 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.258925    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62b89f9f-c9e9-420f-9cec-c33094408036-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.259004 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.258978    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.259645 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.259622    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.259812 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.259647    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.259952 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.259793    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.261526 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.261504    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62b89f9f-c9e9-420f-9cec-c33094408036-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.261709 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.261688    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62b89f9f-c9e9-420f-9cec-c33094408036-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.273328 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.273298    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9g6d\" (UniqueName: \"kubernetes.io/projected/62b89f9f-c9e9-420f-9cec-c33094408036-kube-api-access-s9g6d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jff25\" (UID: \"62b89f9f-c9e9-420f-9cec-c33094408036\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.426036 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.425950    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:34.561367 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:34.561341    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"]
Jul 09 03:52:34.563245 ip-10-0-134-164 kubenswrapper[2582]: W0709 03:52:34.563219    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b89f9f_c9e9_420f_9cec_c33094408036.slice/crio-21e5651ed811a3d0a7e0557ae8233124fcb9cc495085ce51a7477c4fda108ddd WatchSource:0}: Error finding container 21e5651ed811a3d0a7e0557ae8233124fcb9cc495085ce51a7477c4fda108ddd: Status 404 returned error can't find the container with id 21e5651ed811a3d0a7e0557ae8233124fcb9cc495085ce51a7477c4fda108ddd
Jul 09 03:52:35.206314 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:35.206276    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25" event={"ID":"62b89f9f-c9e9-420f-9cec-c33094408036","Type":"ContainerStarted","Data":"08701ec96c761b3c0f5bff3f5a66af8e16282199e85292af69d0b12b9a8abafc"}
Jul 09 03:52:35.206314 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:35.206335    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25" event={"ID":"62b89f9f-c9e9-420f-9cec-c33094408036","Type":"ContainerStarted","Data":"21e5651ed811a3d0a7e0557ae8233124fcb9cc495085ce51a7477c4fda108ddd"}
Jul 09 03:52:35.224371 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:35.224314    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25" podStartSLOduration=1.05290438 podStartE2EDuration="1.224297024s" totalImagesPullingTime="171.392644ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:52:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:52:35.221666258 +0000 UTC m=+572.040922036" watchObservedRunningTime="2026-07-09 03:52:35.224297024 +0000 UTC m=+572.043552801"
Jul 09 03:52:44.426507 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:44.426467    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:44.426942 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:44.426549    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:44.439109 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:44.439081    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:45.253069 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:45.253027    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:52:45.264495 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:45.264467    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jff25"
Jul 09 03:52:55.747557 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:52:55.747515    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:53:03.588290 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:03.588252    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 03:53:03.591337 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:03.591313    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 03:53:03.591955 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:03.591933    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 03:53:03.594517 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:03.594498    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 03:53:06.365301 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:06.365261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:53:17.108420 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:17.108378    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:53:17.209593 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:17.209554    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:53:28.788353 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:28.788262    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:53:48.713362 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:48.713321    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:53:48.811080 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:48.811043    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:53:49.845898 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:49.845848    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:53:49.851515 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:49.851483    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5557679ddc-pss97"]
Jul 09 03:53:49.851699 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:49.851677    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5557679ddc-pss97" podUID="d40caca6-8071-4a14-98cc-d93f17628ccc" containerName="authorino" containerID="cri-o://71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3" gracePeriod=30
Jul 09 03:53:49.933767 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:49.933730    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:53:50.102585 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.102519    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5557679ddc-pss97"
Jul 09 03:53:50.208354 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.208324    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d40caca6-8071-4a14-98cc-d93f17628ccc-tls-cert\") pod \"d40caca6-8071-4a14-98cc-d93f17628ccc\" (UID: \"d40caca6-8071-4a14-98cc-d93f17628ccc\") "
Jul 09 03:53:50.208538 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.208392    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8mck\" (UniqueName: \"kubernetes.io/projected/d40caca6-8071-4a14-98cc-d93f17628ccc-kube-api-access-r8mck\") pod \"d40caca6-8071-4a14-98cc-d93f17628ccc\" (UID: \"d40caca6-8071-4a14-98cc-d93f17628ccc\") "
Jul 09 03:53:50.210629 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.210596    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d40caca6-8071-4a14-98cc-d93f17628ccc-kube-api-access-r8mck" (OuterVolumeSpecName: "kube-api-access-r8mck") pod "d40caca6-8071-4a14-98cc-d93f17628ccc" (UID: "d40caca6-8071-4a14-98cc-d93f17628ccc"). InnerVolumeSpecName "kube-api-access-r8mck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:53:50.218080 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.218051    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d40caca6-8071-4a14-98cc-d93f17628ccc-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d40caca6-8071-4a14-98cc-d93f17628ccc" (UID: "d40caca6-8071-4a14-98cc-d93f17628ccc"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 03:53:50.309426 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.309389    2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d40caca6-8071-4a14-98cc-d93f17628ccc-tls-cert\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:53:50.309426 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.309417    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8mck\" (UniqueName: \"kubernetes.io/projected/d40caca6-8071-4a14-98cc-d93f17628ccc-kube-api-access-r8mck\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:53:50.570389 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.570348    2582 generic.go:356] "Generic (PLEG): container finished" podID="d40caca6-8071-4a14-98cc-d93f17628ccc" containerID="71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3" exitCode=0
Jul 09 03:53:50.570601 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.570410    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5557679ddc-pss97"
Jul 09 03:53:50.570601 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.570421    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5557679ddc-pss97" event={"ID":"d40caca6-8071-4a14-98cc-d93f17628ccc","Type":"ContainerDied","Data":"71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3"}
Jul 09 03:53:50.570601 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.570453    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5557679ddc-pss97" event={"ID":"d40caca6-8071-4a14-98cc-d93f17628ccc","Type":"ContainerDied","Data":"c3483c5ed209154e179ce8feda8b5131792ef250da4eafa03838937c5b0275fa"}
Jul 09 03:53:50.570601 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.570471    2582 scope.go:117] "RemoveContainer" containerID="71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3"
Jul 09 03:53:50.579430 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.579408    2582 scope.go:117] "RemoveContainer" containerID="71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3"
Jul 09 03:53:50.579758 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:53:50.579734    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3\": container with ID starting with 71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3 not found: ID does not exist" containerID="71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3"
Jul 09 03:53:50.579831 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.579771    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3"} err="failed to get container status \"71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3\": rpc error: code = NotFound desc = could not find container \"71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3\": container with ID starting with 71ab77ff4a3e0b99bdca8179ceffe5d8c95c3955f0d5dd0d15b72b27b7d09ba3 not found: ID does not exist"
Jul 09 03:53:50.591044 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.591017    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5557679ddc-pss97"]
Jul 09 03:53:50.595216 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:50.595173    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5557679ddc-pss97"]
Jul 09 03:53:51.404387 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:51.404346    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:53:51.713905 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:53:51.713814    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d40caca6-8071-4a14-98cc-d93f17628ccc" path="/var/lib/kubelet/pods/d40caca6-8071-4a14-98cc-d93f17628ccc/volumes"
Jul 09 03:54:59.422628 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:54:59.422545    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:55:01.408167 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:55:01.408126    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:55:39.892091 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:55:39.892049    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:55:39.994443 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:55:39.994406    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:55:40.108917 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:55:40.108855    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:55:45.641970 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:55:45.641932    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:55:45.748449 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:55:45.748411    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:55:45.842298 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:55:45.842254    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:55:45.977928 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:55:45.977816    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:55:51.583952 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:55:51.583917    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:55:51.683984 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:55:51.683949    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:58:03.620707 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:58:03.620679    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 03:58:03.621328 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:58:03.621136    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 03:58:03.623288 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:58:03.623266    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 03:58:03.623796 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:58:03.623777    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 03:58:38.911090 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:58:38.911053    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 03:59:42.760793 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:42.760703    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"]
Jul 09 03:59:42.761311 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:42.760998    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz" podUID="d6e7f83b-4b84-43b4-bb47-bd2a7d97b446" containerName="manager" containerID="cri-o://86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2" gracePeriod=10
Jul 09 03:59:44.823156 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:44.823125    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:59:44.825153 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:44.825124    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-extensions-socket-volume\") pod \"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446\" (UID: \"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446\") "
Jul 09 03:59:44.825226 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:44.825168    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gv7d8\" (UniqueName: \"kubernetes.io/projected/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-kube-api-access-gv7d8\") pod \"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446\" (UID: \"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446\") "
Jul 09 03:59:44.825538 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:44.825514    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d6e7f83b-4b84-43b4-bb47-bd2a7d97b446" (UID: "d6e7f83b-4b84-43b4-bb47-bd2a7d97b446"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:59:44.827234 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:44.827208    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-kube-api-access-gv7d8" (OuterVolumeSpecName: "kube-api-access-gv7d8") pod "d6e7f83b-4b84-43b4-bb47-bd2a7d97b446" (UID: "d6e7f83b-4b84-43b4-bb47-bd2a7d97b446"). InnerVolumeSpecName "kube-api-access-gv7d8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:59:44.926526 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:44.926445    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-extensions-socket-volume\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:59:44.926526 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:44.926473    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv7d8\" (UniqueName: \"kubernetes.io/projected/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446-kube-api-access-gv7d8\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 03:59:45.294150 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:45.294121    2582 generic.go:356] "Generic (PLEG): container finished" podID="d6e7f83b-4b84-43b4-bb47-bd2a7d97b446" containerID="86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2" exitCode=0
Jul 09 03:59:45.294333 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:45.294169    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"
Jul 09 03:59:45.294333 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:45.294205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz" event={"ID":"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446","Type":"ContainerDied","Data":"86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2"}
Jul 09 03:59:45.294333 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:45.294257    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz" event={"ID":"d6e7f83b-4b84-43b4-bb47-bd2a7d97b446","Type":"ContainerDied","Data":"95401e0daa980578fd90d5f7af1fcf271be297c90d6fd3fc2c049fa135af9ba7"}
Jul 09 03:59:45.294333 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:45.294278    2582 scope.go:117] "RemoveContainer" containerID="86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2"
Jul 09 03:59:45.304636 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:45.304617    2582 scope.go:117] "RemoveContainer" containerID="86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2"
Jul 09 03:59:45.304967 ip-10-0-134-164 kubenswrapper[2582]: E0709 03:59:45.304946    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2\": container with ID starting with 86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2 not found: ID does not exist" containerID="86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2"
Jul 09 03:59:45.305029 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:45.304976    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2"} err="failed to get container status \"86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2\": rpc error: code = NotFound desc = could not find container \"86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2\": container with ID starting with 86af387a66c7b3b04907a00a6648c8338a61c1a5a83aa9ca06f31cbc877617a2 not found: ID does not exist"
Jul 09 03:59:45.315578 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:45.315544    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"]
Jul 09 03:59:45.318838 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:45.318812    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4fsxz"]
Jul 09 03:59:45.713666 ip-10-0-134-164 kubenswrapper[2582]: I0709 03:59:45.713584    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e7f83b-4b84-43b4-bb47-bd2a7d97b446" path="/var/lib/kubelet/pods/d6e7f83b-4b84-43b4-bb47-bd2a7d97b446/volumes"
Jul 09 04:00:00.142216 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.142182    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"]
Jul 09 04:00:00.142631 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.142455    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d40caca6-8071-4a14-98cc-d93f17628ccc" containerName="authorino"
Jul 09 04:00:00.142631 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.142466    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d40caca6-8071-4a14-98cc-d93f17628ccc" containerName="authorino"
Jul 09 04:00:00.142631 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.142485    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e7f83b-4b84-43b4-bb47-bd2a7d97b446" containerName="manager"
Jul 09 04:00:00.142631 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.142490    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e7f83b-4b84-43b4-bb47-bd2a7d97b446" containerName="manager"
Jul 09 04:00:00.142631 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.142539    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d40caca6-8071-4a14-98cc-d93f17628ccc" containerName="authorino"
Jul 09 04:00:00.142631 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.142550    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e7f83b-4b84-43b4-bb47-bd2a7d97b446" containerName="manager"
Jul 09 04:00:00.150718 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.150689    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"
Jul 09 04:00:00.152128 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.152095    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"]
Jul 09 04:00:00.153555 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.153531    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 04:00:00.153694 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.153531    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kmjr\""
Jul 09 04:00:00.154659 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.154637    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 04:00:00.232378 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.232338    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n4x2\" (UniqueName: \"kubernetes.io/projected/82fd3509-d590-4ece-9a7a-cc26d2392534-kube-api-access-9n4x2\") pod \"maas-api-key-cleanup-29726160-7b7jd\" (UID: \"82fd3509-d590-4ece-9a7a-cc26d2392534\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"
Jul 09 04:00:00.333028 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.332986    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n4x2\" (UniqueName: \"kubernetes.io/projected/82fd3509-d590-4ece-9a7a-cc26d2392534-kube-api-access-9n4x2\") pod \"maas-api-key-cleanup-29726160-7b7jd\" (UID: \"82fd3509-d590-4ece-9a7a-cc26d2392534\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"
Jul 09 04:00:00.341442 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.341411    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n4x2\" (UniqueName: \"kubernetes.io/projected/82fd3509-d590-4ece-9a7a-cc26d2392534-kube-api-access-9n4x2\") pod \"maas-api-key-cleanup-29726160-7b7jd\" (UID: \"82fd3509-d590-4ece-9a7a-cc26d2392534\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"
Jul 09 04:00:00.462058 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.461967    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"
Jul 09 04:00:00.595612 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.595587    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"]
Jul 09 04:00:00.597775 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:00:00.597746    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82fd3509_d590_4ece_9a7a_cc26d2392534.slice/crio-5b064634594d9866a3be80c6ad012197abde49116d4954da3649732f213f68b2 WatchSource:0}: Error finding container 5b064634594d9866a3be80c6ad012197abde49116d4954da3649732f213f68b2: Status 404 returned error can't find the container with id 5b064634594d9866a3be80c6ad012197abde49116d4954da3649732f213f68b2
Jul 09 04:00:00.599744 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.599722    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 04:00:00.863267 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:00.863233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:01.373337 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:01.373299    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd" event={"ID":"82fd3509-d590-4ece-9a7a-cc26d2392534","Type":"ContainerStarted","Data":"5b064634594d9866a3be80c6ad012197abde49116d4954da3649732f213f68b2"}
Jul 09 04:00:18.463098 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:18.462765    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd" event={"ID":"82fd3509-d590-4ece-9a7a-cc26d2392534","Type":"ContainerStarted","Data":"2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a"}
Jul 09 04:00:18.479837 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:18.479783    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd" podStartSLOduration=1.226261056 podStartE2EDuration="18.479767636s" totalImagesPullingTime="17.25350658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:18.478259282 +0000 UTC m=+1035.297515062" watchObservedRunningTime="2026-07-09 04:00:18.479767636 +0000 UTC m=+1035.299023413"
Jul 09 04:00:22.380937 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:22.380896    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:00:43.916963 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:43.916922    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"]
Jul 09 04:00:43.920143 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:43.920124    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"
Jul 09 04:00:43.924768 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:43.924742    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 04:00:43.925074 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:43.925057    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 04:00:43.925319 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:43.925302    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fhkmc\""
Jul 09 04:00:43.938388 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:43.938357    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"]
Jul 09 04:00:43.995188 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:43.995148    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/362aa580-7601-490e-8a64-b444c4019e02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cbh2z\" (UID: \"362aa580-7601-490e-8a64-b444c4019e02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"
Jul 09 04:00:43.995385 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:43.995212    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmlrd\" (UniqueName: \"kubernetes.io/projected/362aa580-7601-490e-8a64-b444c4019e02-kube-api-access-nmlrd\") pod \"kuadrant-operator-controller-manager-86878c769f-cbh2z\" (UID: \"362aa580-7601-490e-8a64-b444c4019e02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"
Jul 09 04:00:44.096153 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:44.096105    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/362aa580-7601-490e-8a64-b444c4019e02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cbh2z\" (UID: \"362aa580-7601-490e-8a64-b444c4019e02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"
Jul 09 04:00:44.096339 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:44.096172    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmlrd\" (UniqueName: \"kubernetes.io/projected/362aa580-7601-490e-8a64-b444c4019e02-kube-api-access-nmlrd\") pod \"kuadrant-operator-controller-manager-86878c769f-cbh2z\" (UID: \"362aa580-7601-490e-8a64-b444c4019e02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"
Jul 09 04:00:44.096585 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:44.096559    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/362aa580-7601-490e-8a64-b444c4019e02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cbh2z\" (UID: \"362aa580-7601-490e-8a64-b444c4019e02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"
Jul 09 04:00:44.105562 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:44.105539    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmlrd\" (UniqueName: \"kubernetes.io/projected/362aa580-7601-490e-8a64-b444c4019e02-kube-api-access-nmlrd\") pod \"kuadrant-operator-controller-manager-86878c769f-cbh2z\" (UID: \"362aa580-7601-490e-8a64-b444c4019e02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"
Jul 09 04:00:44.231217 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:44.231114    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"
Jul 09 04:00:44.374905 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:44.374877    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"]
Jul 09 04:00:44.377289 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:00:44.377257    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod362aa580_7601_490e_8a64_b444c4019e02.slice/crio-0e093efed6150b46901db49cae5443cea91f932e9685777b5a159d292d73baa3 WatchSource:0}: Error finding container 0e093efed6150b46901db49cae5443cea91f932e9685777b5a159d292d73baa3: Status 404 returned error can't find the container with id 0e093efed6150b46901db49cae5443cea91f932e9685777b5a159d292d73baa3
Jul 09 04:00:44.587990 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:44.587949    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z" event={"ID":"362aa580-7601-490e-8a64-b444c4019e02","Type":"ContainerStarted","Data":"00265f0eceb6254dfc4ac70f85ba73edb07d484189051c88229c70cc2d63b4f9"}
Jul 09 04:00:44.588149 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:44.588000    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z" event={"ID":"362aa580-7601-490e-8a64-b444c4019e02","Type":"ContainerStarted","Data":"0e093efed6150b46901db49cae5443cea91f932e9685777b5a159d292d73baa3"}
Jul 09 04:00:44.588149 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:44.588102    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"
Jul 09 04:00:44.617857 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:44.617801    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z" podStartSLOduration=1.617782902 podStartE2EDuration="1.617782902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:00:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:44.614567427 +0000 UTC m=+1061.433823204" watchObservedRunningTime="2026-07-09 04:00:44.617782902 +0000 UTC m=+1061.437038678"
Jul 09 04:00:55.594945 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:55.594915    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbh2z"
Jul 09 04:00:57.721294 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:00:57.721259    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:01:00.016585 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:00.016549    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"]
Jul 09 04:01:00.017010 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:00.016747    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd" podUID="82fd3509-d590-4ece-9a7a-cc26d2392534" containerName="cleanup" containerID="cri-o://2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a" gracePeriod=30
Jul 09 04:01:30.212856 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.212824    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"
Jul 09 04:01:30.290931 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.290901    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9n4x2\" (UniqueName: \"kubernetes.io/projected/82fd3509-d590-4ece-9a7a-cc26d2392534-kube-api-access-9n4x2\") pod \"82fd3509-d590-4ece-9a7a-cc26d2392534\" (UID: \"82fd3509-d590-4ece-9a7a-cc26d2392534\") "
Jul 09 04:01:30.293170 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.293123    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82fd3509-d590-4ece-9a7a-cc26d2392534-kube-api-access-9n4x2" (OuterVolumeSpecName: "kube-api-access-9n4x2") pod "82fd3509-d590-4ece-9a7a-cc26d2392534" (UID: "82fd3509-d590-4ece-9a7a-cc26d2392534"). InnerVolumeSpecName "kube-api-access-9n4x2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:01:30.392319 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.392279    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n4x2\" (UniqueName: \"kubernetes.io/projected/82fd3509-d590-4ece-9a7a-cc26d2392534-kube-api-access-9n4x2\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:01:30.813510 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.813472    2582 generic.go:356] "Generic (PLEG): container finished" podID="82fd3509-d590-4ece-9a7a-cc26d2392534" containerID="2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a" exitCode=137
Jul 09 04:01:30.813720 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.813523    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"
Jul 09 04:01:30.813720 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.813558    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd" event={"ID":"82fd3509-d590-4ece-9a7a-cc26d2392534","Type":"ContainerDied","Data":"2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a"}
Jul 09 04:01:30.813720 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.813612    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd" event={"ID":"82fd3509-d590-4ece-9a7a-cc26d2392534","Type":"ContainerDied","Data":"5b064634594d9866a3be80c6ad012197abde49116d4954da3649732f213f68b2"}
Jul 09 04:01:30.813720 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.813636    2582 scope.go:117] "RemoveContainer" containerID="2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a"
Jul 09 04:01:30.822779 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.822762    2582 scope.go:117] "RemoveContainer" containerID="2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a"
Jul 09 04:01:30.823100 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:01:30.823081    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a\": container with ID starting with 2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a not found: ID does not exist" containerID="2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a"
Jul 09 04:01:30.823149 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.823109    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a"} err="failed to get container status \"2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a\": rpc error: code = NotFound desc = could not find container \"2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a\": container with ID starting with 2a503f73b7db5e35b48c3bb8bb7a20e8a94b479b6f1a7b701d48f90dbf250d2a not found: ID does not exist"
Jul 09 04:01:30.837746 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.837717    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"]
Jul 09 04:01:30.842140 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:30.842116    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726160-7b7jd"]
Jul 09 04:01:31.718956 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:01:31.718918    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82fd3509-d590-4ece-9a7a-cc26d2392534" path="/var/lib/kubelet/pods/82fd3509-d590-4ece-9a7a-cc26d2392534/volumes"
Jul 09 04:03:03.651882 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:03:03.651832    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 04:03:03.652404 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:03:03.651999    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 04:03:03.654359 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:03:03.654339    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 04:03:03.654359 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:03:03.654349    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 04:06:43.839939 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:06:43.839899    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:50.339824 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:06:50.339785    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:06:58.593650 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:06:58.593543    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:02.328645 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:02.328604    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:03.747721 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:03.747689    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:06.502732 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:06.502689    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:06.602154 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:06.602114    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:06.952603 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:06.952565    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:12.480389 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:12.480343    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:12.587220 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:12.587180    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:12.696960 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:12.696914    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:12.792719 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:12.792681    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:13.907054 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:13.907021    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:13.938372 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:13.938334    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"]
Jul 09 04:07:13.938795 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:13.938769    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82fd3509-d590-4ece-9a7a-cc26d2392534" containerName="cleanup"
Jul 09 04:07:13.938795 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:13.938796    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="82fd3509-d590-4ece-9a7a-cc26d2392534" containerName="cleanup"
Jul 09 04:07:13.938993 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:13.938900    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="82fd3509-d590-4ece-9a7a-cc26d2392534" containerName="cleanup"
Jul 09 04:07:13.941978 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:13.941955    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:13.944748 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:13.944720    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 04:07:13.944910 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:13.944742    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1c6bc5d3-serving-cert\""
Jul 09 04:07:13.944910 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:13.944752    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kmjr\""
Jul 09 04:07:13.945776 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:13.945760    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 04:07:13.953057 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:13.953032    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"]
Jul 09 04:07:14.005249 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.005194    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:14.034202 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.034167    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8762daea-767b-4d94-9dc8-551809bbae73-maas-api-tls\") pod \"maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:14.034367 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.034210    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8762daea-767b-4d94-9dc8-551809bbae73-openshift-service-ca\") pod \"maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:14.034367 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.034230    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9l9d\" (UniqueName: \"kubernetes.io/projected/8762daea-767b-4d94-9dc8-551809bbae73-kube-api-access-n9l9d\") pod \"maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:14.135124 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.135080    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l9d\" (UniqueName: \"kubernetes.io/projected/8762daea-767b-4d94-9dc8-551809bbae73-kube-api-access-n9l9d\") pod \"maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:14.135333 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.135201    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8762daea-767b-4d94-9dc8-551809bbae73-maas-api-tls\") pod \"maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:14.135333 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.135231    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8762daea-767b-4d94-9dc8-551809bbae73-openshift-service-ca\") pod \"maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:14.135449 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:07:14.135341    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-serving-cert: secret "maas-api-e2e-ait-1c6bc5d3-serving-cert" not found
Jul 09 04:07:14.135449 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:07:14.135411    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8762daea-767b-4d94-9dc8-551809bbae73-maas-api-tls podName:8762daea-767b-4d94-9dc8-551809bbae73 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:14.635393225 +0000 UTC m=+1451.454648980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8762daea-767b-4d94-9dc8-551809bbae73-maas-api-tls") pod "maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk" (UID: "8762daea-767b-4d94-9dc8-551809bbae73") : secret "maas-api-e2e-ait-1c6bc5d3-serving-cert" not found
Jul 09 04:07:14.135960 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.135939    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8762daea-767b-4d94-9dc8-551809bbae73-openshift-service-ca\") pod \"maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:14.145084 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.145059    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9l9d\" (UniqueName: \"kubernetes.io/projected/8762daea-767b-4d94-9dc8-551809bbae73-kube-api-access-n9l9d\") pod \"maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:14.316940 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.316899    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:14.495098 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.495060    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:14.609336 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.609250    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:14.640244 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.640205    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8762daea-767b-4d94-9dc8-551809bbae73-maas-api-tls\") pod \"maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:14.642657 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.642638    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8762daea-767b-4d94-9dc8-551809bbae73-maas-api-tls\") pod \"maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:14.853852 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.853806    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:14.998194 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:14.998151    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"]
Jul 09 04:07:15.006921 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:07:15.006890    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8762daea_767b_4d94_9dc8_551809bbae73.slice/crio-cbc78b1d5a0ae4e130daa8c36e8171354814ea2218fd725a8a77117de5eeff40 WatchSource:0}: Error finding container cbc78b1d5a0ae4e130daa8c36e8171354814ea2218fd725a8a77117de5eeff40: Status 404 returned error can't find the container with id cbc78b1d5a0ae4e130daa8c36e8171354814ea2218fd725a8a77117de5eeff40
Jul 09 04:07:15.008559 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:15.008538    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 04:07:15.505267 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:15.505231    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk" event={"ID":"8762daea-767b-4d94-9dc8-551809bbae73","Type":"ContainerStarted","Data":"cbc78b1d5a0ae4e130daa8c36e8171354814ea2218fd725a8a77117de5eeff40"}
Jul 09 04:07:16.239018 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:16.238772    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:17.515677 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:17.515639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk" event={"ID":"8762daea-767b-4d94-9dc8-551809bbae73","Type":"ContainerStarted","Data":"f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712"}
Jul 09 04:07:17.516107 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:17.515734    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:17.531401 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:17.531347    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk" podStartSLOduration=2.771628529 podStartE2EDuration="4.531330288s" totalImagesPullingTime="1.759701759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:17.530338538 +0000 UTC m=+1454.349594320" watchObservedRunningTime="2026-07-09 04:07:17.531330288 +0000 UTC m=+1454.350586064"
Jul 09 04:07:18.896645 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:18.896615    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:19.074569 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.074520    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:19.175885 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.175792    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:19.238115 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.238080    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"]
Jul 09 04:07:19.276248 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.276212    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:19.525386 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.525294    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk" podUID="8762daea-767b-4d94-9dc8-551809bbae73" containerName="maas-api" containerID="cri-o://f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712" gracePeriod=30
Jul 09 04:07:19.772451 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.772425    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:19.887289 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.887254    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9l9d\" (UniqueName: \"kubernetes.io/projected/8762daea-767b-4d94-9dc8-551809bbae73-kube-api-access-n9l9d\") pod \"8762daea-767b-4d94-9dc8-551809bbae73\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") "
Jul 09 04:07:19.887511 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.887323    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8762daea-767b-4d94-9dc8-551809bbae73-maas-api-tls\") pod \"8762daea-767b-4d94-9dc8-551809bbae73\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") "
Jul 09 04:07:19.887511 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.887360    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8762daea-767b-4d94-9dc8-551809bbae73-openshift-service-ca\") pod \"8762daea-767b-4d94-9dc8-551809bbae73\" (UID: \"8762daea-767b-4d94-9dc8-551809bbae73\") "
Jul 09 04:07:19.887721 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.887698    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8762daea-767b-4d94-9dc8-551809bbae73-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8762daea-767b-4d94-9dc8-551809bbae73" (UID: "8762daea-767b-4d94-9dc8-551809bbae73"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:07:19.889494 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.889461    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8762daea-767b-4d94-9dc8-551809bbae73-kube-api-access-n9l9d" (OuterVolumeSpecName: "kube-api-access-n9l9d") pod "8762daea-767b-4d94-9dc8-551809bbae73" (UID: "8762daea-767b-4d94-9dc8-551809bbae73"). InnerVolumeSpecName "kube-api-access-n9l9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:07:19.889595 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.889572    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8762daea-767b-4d94-9dc8-551809bbae73-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8762daea-767b-4d94-9dc8-551809bbae73" (UID: "8762daea-767b-4d94-9dc8-551809bbae73"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:07:19.988055 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.988015    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9l9d\" (UniqueName: \"kubernetes.io/projected/8762daea-767b-4d94-9dc8-551809bbae73-kube-api-access-n9l9d\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:07:19.988055 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.988050    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8762daea-767b-4d94-9dc8-551809bbae73-maas-api-tls\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:07:19.988055 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:19.988060    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8762daea-767b-4d94-9dc8-551809bbae73-openshift-service-ca\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:07:20.391809 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:20.391772    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:20.530873 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:20.530840    2582 generic.go:356] "Generic (PLEG): container finished" podID="8762daea-767b-4d94-9dc8-551809bbae73" containerID="f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712" exitCode=0
Jul 09 04:07:20.531075 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:20.530910    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"
Jul 09 04:07:20.531075 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:20.530932    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk" event={"ID":"8762daea-767b-4d94-9dc8-551809bbae73","Type":"ContainerDied","Data":"f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712"}
Jul 09 04:07:20.531075 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:20.530993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk" event={"ID":"8762daea-767b-4d94-9dc8-551809bbae73","Type":"ContainerDied","Data":"cbc78b1d5a0ae4e130daa8c36e8171354814ea2218fd725a8a77117de5eeff40"}
Jul 09 04:07:20.531075 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:20.531027    2582 scope.go:117] "RemoveContainer" containerID="f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712"
Jul 09 04:07:20.540663 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:20.540643    2582 scope.go:117] "RemoveContainer" containerID="f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712"
Jul 09 04:07:20.540978 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:07:20.540961    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712\": container with ID starting with f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712 not found: ID does not exist" containerID="f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712"
Jul 09 04:07:20.541016 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:20.540988    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712"} err="failed to get container status \"f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712\": rpc error: code = NotFound desc = could not find container \"f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712\": container with ID starting with f925a1eb759ef24f8416bc92ccc3fff463f711db6c4d67c12ea4bc04dd14b712 not found: ID does not exist"
Jul 09 04:07:20.552338 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:20.552305    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"]
Jul 09 04:07:20.555724 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:20.555700    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c6bc5d3-6d4c445774-m8wnk"]
Jul 09 04:07:21.714480 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:21.714445    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8762daea-767b-4d94-9dc8-551809bbae73" path="/var/lib/kubelet/pods/8762daea-767b-4d94-9dc8-551809bbae73/volumes"
Jul 09 04:07:25.057331 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:25.057286    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:25.113700 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:25.113665    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:25.264481 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:25.264432    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:25.363731 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:25.363635    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:26.364761 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.364716    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:26.446783 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.446743    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"]
Jul 09 04:07:26.447056 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.447043    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8762daea-767b-4d94-9dc8-551809bbae73" containerName="maas-api"
Jul 09 04:07:26.447056 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.447056    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8762daea-767b-4d94-9dc8-551809bbae73" containerName="maas-api"
Jul 09 04:07:26.447132 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.447110    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8762daea-767b-4d94-9dc8-551809bbae73" containerName="maas-api"
Jul 09 04:07:26.451137 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.451117    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:26.455391 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.455360    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5294925d-serving-cert\""
Jul 09 04:07:26.455540 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.455423    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 04:07:26.455655 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.455639    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kmjr\""
Jul 09 04:07:26.456571 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.456552    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 04:07:26.466438 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.466409    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"]
Jul 09 04:07:26.473105 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.473074    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:26.615176 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.615075    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:26.643221 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.643182    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-maas-api-tls\") pod \"maas-api-e2e-ait-5294925d-6fcd66db8f-597lx\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:26.643221 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.643221    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-openshift-service-ca\") pod \"maas-api-e2e-ait-5294925d-6fcd66db8f-597lx\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:26.643456 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.643242    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgbg2\" (UniqueName: \"kubernetes.io/projected/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-kube-api-access-cgbg2\") pod \"maas-api-e2e-ait-5294925d-6fcd66db8f-597lx\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:26.744360 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.744318    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-maas-api-tls\") pod \"maas-api-e2e-ait-5294925d-6fcd66db8f-597lx\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:26.744360 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.744361    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-openshift-service-ca\") pod \"maas-api-e2e-ait-5294925d-6fcd66db8f-597lx\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:26.744360 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.744378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgbg2\" (UniqueName: \"kubernetes.io/projected/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-kube-api-access-cgbg2\") pod \"maas-api-e2e-ait-5294925d-6fcd66db8f-597lx\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:26.744631 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:07:26.744554    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-serving-cert: secret "maas-api-e2e-ait-5294925d-serving-cert" not found
Jul 09 04:07:26.744631 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:07:26.744620    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-maas-api-tls podName:b0c5b4da-38a4-46bc-8950-c0f0cf947da0 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:27.244604528 +0000 UTC m=+1464.063860284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-maas-api-tls") pod "maas-api-e2e-ait-5294925d-6fcd66db8f-597lx" (UID: "b0c5b4da-38a4-46bc-8950-c0f0cf947da0") : secret "maas-api-e2e-ait-5294925d-serving-cert" not found
Jul 09 04:07:26.745105 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.745088    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-openshift-service-ca\") pod \"maas-api-e2e-ait-5294925d-6fcd66db8f-597lx\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:26.757023 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.756989    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgbg2\" (UniqueName: \"kubernetes.io/projected/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-kube-api-access-cgbg2\") pod \"maas-api-e2e-ait-5294925d-6fcd66db8f-597lx\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:26.932033 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:26.931938    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:27.044065 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:27.044030    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:27.144225 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:27.144193    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:27.249129 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:27.249038    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-maas-api-tls\") pod \"maas-api-e2e-ait-5294925d-6fcd66db8f-597lx\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:27.251575 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:27.251546    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-maas-api-tls\") pod \"maas-api-e2e-ait-5294925d-6fcd66db8f-597lx\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:27.361205 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:27.361157    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:27.492034 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:27.491998    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"]
Jul 09 04:07:27.495106 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:07:27.495071    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c5b4da_38a4_46bc_8950_c0f0cf947da0.slice/crio-35dfae8231d368ac69553f4de4b2a5857f517be3bcc29fdc78d10ff23b13cd7b WatchSource:0}: Error finding container 35dfae8231d368ac69553f4de4b2a5857f517be3bcc29fdc78d10ff23b13cd7b: Status 404 returned error can't find the container with id 35dfae8231d368ac69553f4de4b2a5857f517be3bcc29fdc78d10ff23b13cd7b
Jul 09 04:07:27.572487 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:27.572454    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx" event={"ID":"b0c5b4da-38a4-46bc-8950-c0f0cf947da0","Type":"ContainerStarted","Data":"35dfae8231d368ac69553f4de4b2a5857f517be3bcc29fdc78d10ff23b13cd7b"}
Jul 09 04:07:28.582555 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:28.582510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx" event={"ID":"b0c5b4da-38a4-46bc-8950-c0f0cf947da0","Type":"ContainerStarted","Data":"694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc"}
Jul 09 04:07:28.583028 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:28.582630    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:28.589515 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:28.589482    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:28.598580 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:28.598532    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx" podStartSLOduration=2.102305852 podStartE2EDuration="2.598515263s" totalImagesPullingTime="496.209411ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:28.597427835 +0000 UTC m=+1465.416683614" watchObservedRunningTime="2026-07-09 04:07:28.598515263 +0000 UTC m=+1465.417771068"
Jul 09 04:07:31.467987 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:31.467952    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:31.870585 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:31.870546    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"]
Jul 09 04:07:31.870836 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:31.870775    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx" podUID="b0c5b4da-38a4-46bc-8950-c0f0cf947da0" containerName="maas-api" containerID="cri-o://694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc" gracePeriod=30
Jul 09 04:07:31.893137 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:31.893054    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:32.116175 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.116146    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:32.190650 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.190561    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-maas-api-tls\") pod \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") "
Jul 09 04:07:32.190650 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.190634    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgbg2\" (UniqueName: \"kubernetes.io/projected/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-kube-api-access-cgbg2\") pod \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") "
Jul 09 04:07:32.190896 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.190659    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-openshift-service-ca\") pod \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\" (UID: \"b0c5b4da-38a4-46bc-8950-c0f0cf947da0\") "
Jul 09 04:07:32.191132 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.191105    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0c5b4da-38a4-46bc-8950-c0f0cf947da0" (UID: "b0c5b4da-38a4-46bc-8950-c0f0cf947da0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:07:32.192701 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.192669    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0c5b4da-38a4-46bc-8950-c0f0cf947da0" (UID: "b0c5b4da-38a4-46bc-8950-c0f0cf947da0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:07:32.192794 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.192737    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-kube-api-access-cgbg2" (OuterVolumeSpecName: "kube-api-access-cgbg2") pod "b0c5b4da-38a4-46bc-8950-c0f0cf947da0" (UID: "b0c5b4da-38a4-46bc-8950-c0f0cf947da0"). InnerVolumeSpecName "kube-api-access-cgbg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:07:32.291422 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.291386    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-maas-api-tls\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:07:32.291422 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.291415    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgbg2\" (UniqueName: \"kubernetes.io/projected/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-kube-api-access-cgbg2\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:07:32.291422 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.291427    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c5b4da-38a4-46bc-8950-c0f0cf947da0-openshift-service-ca\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:07:32.604270 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.604234    2582 generic.go:356] "Generic (PLEG): container finished" podID="b0c5b4da-38a4-46bc-8950-c0f0cf947da0" containerID="694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc" exitCode=0
Jul 09 04:07:32.604744 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.604288    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"
Jul 09 04:07:32.604744 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.604318    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx" event={"ID":"b0c5b4da-38a4-46bc-8950-c0f0cf947da0","Type":"ContainerDied","Data":"694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc"}
Jul 09 04:07:32.604744 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.604372    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx" event={"ID":"b0c5b4da-38a4-46bc-8950-c0f0cf947da0","Type":"ContainerDied","Data":"35dfae8231d368ac69553f4de4b2a5857f517be3bcc29fdc78d10ff23b13cd7b"}
Jul 09 04:07:32.604744 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.604396    2582 scope.go:117] "RemoveContainer" containerID="694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc"
Jul 09 04:07:32.613886 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.613835    2582 scope.go:117] "RemoveContainer" containerID="694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc"
Jul 09 04:07:32.614148 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:07:32.614129    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc\": container with ID starting with 694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc not found: ID does not exist" containerID="694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc"
Jul 09 04:07:32.614217 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.614157    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc"} err="failed to get container status \"694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc\": rpc error: code = NotFound desc = could not find container \"694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc\": container with ID starting with 694ceca44d00e02e36c4682c54fa531b1e6f12ad681407cd3f4aae0e2a6474bc not found: ID does not exist"
Jul 09 04:07:32.626027 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.625998    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"]
Jul 09 04:07:32.627793 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:32.627774    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5294925d-6fcd66db8f-597lx"]
Jul 09 04:07:33.714516 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:33.714477    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0c5b4da-38a4-46bc-8950-c0f0cf947da0" path="/var/lib/kubelet/pods/b0c5b4da-38a4-46bc-8950-c0f0cf947da0/volumes"
Jul 09 04:07:37.517564 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:37.517525    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:37.623292 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:37.623258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:39.769986 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:39.769950    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:43.439198 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:43.439156    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:43.497615 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:43.497567    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:43.712063 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:43.711977    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:43.822800 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:43.822759    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:44.850389 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.850357    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:44.921521 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.921487    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"]
Jul 09 04:07:44.921873 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.921847    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0c5b4da-38a4-46bc-8950-c0f0cf947da0" containerName="maas-api"
Jul 09 04:07:44.922008 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.921882    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0c5b4da-38a4-46bc-8950-c0f0cf947da0" containerName="maas-api"
Jul 09 04:07:44.922008 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.921951    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0c5b4da-38a4-46bc-8950-c0f0cf947da0" containerName="maas-api"
Jul 09 04:07:44.924928 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.924878    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:44.926158 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.926141    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:44.934150 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.934127    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kmjr\""
Jul 09 04:07:44.934278 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.934170    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 04:07:44.934568 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.934490    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-0a452316-serving-cert\""
Jul 09 04:07:44.934568 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.934500    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 04:07:44.935507 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.935484    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"]
Jul 09 04:07:44.987575 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:44.987536    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:45.001174 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.001134    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/972a233e-5587-4efd-8619-0bc42415348d-openshift-service-ca\") pod \"maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:45.001363 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.001224    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/972a233e-5587-4efd-8619-0bc42415348d-maas-api-tls\") pod \"maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:45.001363 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.001261    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlbq5\" (UniqueName: \"kubernetes.io/projected/972a233e-5587-4efd-8619-0bc42415348d-kube-api-access-jlbq5\") pod \"maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:45.102652 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.102558    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/972a233e-5587-4efd-8619-0bc42415348d-maas-api-tls\") pod \"maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:45.102652 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.102626    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlbq5\" (UniqueName: \"kubernetes.io/projected/972a233e-5587-4efd-8619-0bc42415348d-kube-api-access-jlbq5\") pod \"maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:45.102848 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.102681    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/972a233e-5587-4efd-8619-0bc42415348d-openshift-service-ca\") pod \"maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:45.102848 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:07:45.102720    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-serving-cert: secret "maas-api-e2e-derive-0a452316-serving-cert" not found
Jul 09 04:07:45.102848 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:07:45.102801    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/972a233e-5587-4efd-8619-0bc42415348d-maas-api-tls podName:972a233e-5587-4efd-8619-0bc42415348d nodeName:}" failed. No retries permitted until 2026-07-09 04:07:45.602779821 +0000 UTC m=+1482.422035584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/972a233e-5587-4efd-8619-0bc42415348d-maas-api-tls") pod "maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x" (UID: "972a233e-5587-4efd-8619-0bc42415348d") : secret "maas-api-e2e-derive-0a452316-serving-cert" not found
Jul 09 04:07:45.103683 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.103660    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/972a233e-5587-4efd-8619-0bc42415348d-openshift-service-ca\") pod \"maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:45.113601 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.113572    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlbq5\" (UniqueName: \"kubernetes.io/projected/972a233e-5587-4efd-8619-0bc42415348d-kube-api-access-jlbq5\") pod \"maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:45.332320 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.332278    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:45.432710 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.432598    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:45.534392 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.534352    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:45.608205 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.608172    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/972a233e-5587-4efd-8619-0bc42415348d-maas-api-tls\") pod \"maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:45.610680 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.610660    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/972a233e-5587-4efd-8619-0bc42415348d-maas-api-tls\") pod \"maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:45.837611 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.837566    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:45.976176 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:45.976151    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"]
Jul 09 04:07:45.978314 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:07:45.978279    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod972a233e_5587_4efd_8619_0bc42415348d.slice/crio-97a74e4cf31ac1f041e265601a8570c82f6a3538639728062e33bcbd6ef05499 WatchSource:0}: Error finding container 97a74e4cf31ac1f041e265601a8570c82f6a3538639728062e33bcbd6ef05499: Status 404 returned error can't find the container with id 97a74e4cf31ac1f041e265601a8570c82f6a3538639728062e33bcbd6ef05499
Jul 09 04:07:46.671367 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:46.671337    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x" event={"ID":"972a233e-5587-4efd-8619-0bc42415348d","Type":"ContainerStarted","Data":"26c88405e8a27efdee736000890707c163e5f771f5e0a97aca5212b470ea4d67"}
Jul 09 04:07:46.671560 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:46.671387    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x" event={"ID":"972a233e-5587-4efd-8619-0bc42415348d","Type":"ContainerStarted","Data":"97a74e4cf31ac1f041e265601a8570c82f6a3538639728062e33bcbd6ef05499"}
Jul 09 04:07:46.671560 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:46.671439    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:46.688853 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:46.688804    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x" podStartSLOduration=2.187257462 podStartE2EDuration="2.688786092s" totalImagesPullingTime="501.52863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:46.687765815 +0000 UTC m=+1483.507021594" watchObservedRunningTime="2026-07-09 04:07:46.688786092 +0000 UTC m=+1483.508041870"
Jul 09 04:07:46.951950 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:46.951912    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:49.220075 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.220037    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:49.420574 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.420545    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:49.523883 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.523764    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:49.525802 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.525775    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"]
Jul 09 04:07:49.526006 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.525978    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x" podUID="972a233e-5587-4efd-8619-0bc42415348d" containerName="maas-api" containerID="cri-o://26c88405e8a27efdee736000890707c163e5f771f5e0a97aca5212b470ea4d67" gracePeriod=30
Jul 09 04:07:49.581283 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.581249    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:49.688992 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.688962    2582 generic.go:356] "Generic (PLEG): container finished" podID="972a233e-5587-4efd-8619-0bc42415348d" containerID="26c88405e8a27efdee736000890707c163e5f771f5e0a97aca5212b470ea4d67" exitCode=0
Jul 09 04:07:49.689245 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.689063    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x" event={"ID":"972a233e-5587-4efd-8619-0bc42415348d","Type":"ContainerDied","Data":"26c88405e8a27efdee736000890707c163e5f771f5e0a97aca5212b470ea4d67"}
Jul 09 04:07:49.769195 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.769161    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:49.845099 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.845056    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/972a233e-5587-4efd-8619-0bc42415348d-maas-api-tls\") pod \"972a233e-5587-4efd-8619-0bc42415348d\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") "
Jul 09 04:07:49.845507 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.845136    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlbq5\" (UniqueName: \"kubernetes.io/projected/972a233e-5587-4efd-8619-0bc42415348d-kube-api-access-jlbq5\") pod \"972a233e-5587-4efd-8619-0bc42415348d\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") "
Jul 09 04:07:49.845507 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.845167    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/972a233e-5587-4efd-8619-0bc42415348d-openshift-service-ca\") pod \"972a233e-5587-4efd-8619-0bc42415348d\" (UID: \"972a233e-5587-4efd-8619-0bc42415348d\") "
Jul 09 04:07:49.845657 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.845558    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/972a233e-5587-4efd-8619-0bc42415348d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "972a233e-5587-4efd-8619-0bc42415348d" (UID: "972a233e-5587-4efd-8619-0bc42415348d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:07:49.847316 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.847294    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/972a233e-5587-4efd-8619-0bc42415348d-kube-api-access-jlbq5" (OuterVolumeSpecName: "kube-api-access-jlbq5") pod "972a233e-5587-4efd-8619-0bc42415348d" (UID: "972a233e-5587-4efd-8619-0bc42415348d"). InnerVolumeSpecName "kube-api-access-jlbq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:07:49.847362 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.847322    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/972a233e-5587-4efd-8619-0bc42415348d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "972a233e-5587-4efd-8619-0bc42415348d" (UID: "972a233e-5587-4efd-8619-0bc42415348d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:07:49.946417 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.946382    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/972a233e-5587-4efd-8619-0bc42415348d-openshift-service-ca\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:07:49.946417 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.946414    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/972a233e-5587-4efd-8619-0bc42415348d-maas-api-tls\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:07:49.946417 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:49.946423    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlbq5\" (UniqueName: \"kubernetes.io/projected/972a233e-5587-4efd-8619-0bc42415348d-kube-api-access-jlbq5\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:07:50.697251 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:50.697222    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"
Jul 09 04:07:50.697684 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:50.697245    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x" event={"ID":"972a233e-5587-4efd-8619-0bc42415348d","Type":"ContainerDied","Data":"97a74e4cf31ac1f041e265601a8570c82f6a3538639728062e33bcbd6ef05499"}
Jul 09 04:07:50.697684 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:50.697318    2582 scope.go:117] "RemoveContainer" containerID="26c88405e8a27efdee736000890707c163e5f771f5e0a97aca5212b470ea4d67"
Jul 09 04:07:50.699891 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:50.699831    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:50.719556 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:50.719530    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"]
Jul 09 04:07:50.722479 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:50.722450    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0a452316-55c4c5dd56-8pd6x"]
Jul 09 04:07:51.713765 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:51.713725    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="972a233e-5587-4efd-8619-0bc42415348d" path="/var/lib/kubelet/pods/972a233e-5587-4efd-8619-0bc42415348d/volumes"
Jul 09 04:07:55.123776 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:55.123739    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:55.190134 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:55.190093    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:55.582630 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:55.582587    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:55.689219 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:55.689178    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:55.783259 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:55.783223    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:07:57.137241 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:07:57.137202    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:00.272418 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:00.272370    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:02.705349 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:02.705303    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:02.806853 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:02.806818    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:02.905635 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:02.905592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:03.008189 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:03.008106    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:03.397704 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:03.397665    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:03.682719 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:03.682628    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 04:08:03.683898 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:03.683856    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 04:08:03.685512 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:03.685490    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 04:08:03.686593 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:03.686573    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 04:08:29.329010 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:29.328973    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:35.138234 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:35.138181    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:35.201215 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:35.201162    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:35.283372 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:35.283341    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:35.383058 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:35.383015    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:51.225813 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:51.225769    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:56.835181 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:56.835140    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:56.962128 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:56.962084    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:08:57.103421 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:08:57.103341    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:07.563039 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:07.562997    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:08.368033 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:08.367987    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:13.968021 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:13.967978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:14.023376 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:14.023338    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:14.111485 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:14.111444    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:14.204591 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:14.204528    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:14.304928 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:14.304894    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:15.017393 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:15.017351    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:15.109418 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:15.109384    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:15.209559 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:15.209527    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:16.745759 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:16.745717    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:22.439842 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:22.439803    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:22.539429 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:22.539387    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:23.097597 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:23.097563    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:28.791528 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:28.791489    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:28.894712 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:28.894664    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:29.017306 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:29.017266    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:29.331110 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:29.331068    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:29.604669 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:29.604581    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:32.174788 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.174749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:32.254896 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.254836    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"]
Jul 09 04:09:32.255314 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.255298    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="972a233e-5587-4efd-8619-0bc42415348d" containerName="maas-api"
Jul 09 04:09:32.255408 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.255318    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="972a233e-5587-4efd-8619-0bc42415348d" containerName="maas-api"
Jul 09 04:09:32.255466 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.255408    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="972a233e-5587-4efd-8619-0bc42415348d" containerName="maas-api"
Jul 09 04:09:32.255739 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.255702    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:32.258526 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.258509    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:32.261610 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.261592    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-99003d6d-serving-cert\""
Jul 09 04:09:32.261734 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.261613    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kmjr\""
Jul 09 04:09:32.261734 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.261663    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 04:09:32.263026 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.263008    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 04:09:32.269516 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.269495    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"]
Jul 09 04:09:32.316343 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.316303    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:32.333445 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.333401    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-maas-api-tls\") pod \"maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:32.333617 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.333451    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z477x\" (UniqueName: \"kubernetes.io/projected/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-kube-api-access-z477x\") pod \"maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:32.333617 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.333481    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-openshift-service-ca\") pod \"maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:32.434651 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.434554    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-maas-api-tls\") pod \"maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:32.434651 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.434600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z477x\" (UniqueName: \"kubernetes.io/projected/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-kube-api-access-z477x\") pod \"maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:32.434651 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.434643    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-openshift-service-ca\") pod \"maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:32.434937 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:09:32.434713    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-serving-cert: secret "maas-api-e2e-mt-99003d6d-serving-cert" not found
Jul 09 04:09:32.434937 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:09:32.434799    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-maas-api-tls podName:fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6 nodeName:}" failed. No retries permitted until 2026-07-09 04:09:32.934781047 +0000 UTC m=+1589.754036813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-maas-api-tls") pod "maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7" (UID: "fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6") : secret "maas-api-e2e-mt-99003d6d-serving-cert" not found
Jul 09 04:09:32.435420 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.435400    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-openshift-service-ca\") pod \"maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:32.444369 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.444346    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z477x\" (UniqueName: \"kubernetes.io/projected/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-kube-api-access-z477x\") pod \"maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:32.526489 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.526449    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:32.864749 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.864706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:32.938801 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.938767    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-maas-api-tls\") pod \"maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:32.941456 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.941417    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-maas-api-tls\") pod \"maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:32.965156 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:32.965120    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:33.170362 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:33.170255    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:33.314844 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:33.314812    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"]
Jul 09 04:09:33.316988 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:09:33.316953    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa4ea91b_faf1_4377_ab4c_3b7ee330f2b6.slice/crio-5c854e36ae0ef2f7f6e5019b547631b8afc5835938c8d6626cc8fe618f315f84 WatchSource:0}: Error finding container 5c854e36ae0ef2f7f6e5019b547631b8afc5835938c8d6626cc8fe618f315f84: Status 404 returned error can't find the container with id 5c854e36ae0ef2f7f6e5019b547631b8afc5835938c8d6626cc8fe618f315f84
Jul 09 04:09:34.197385 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:34.197347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7" event={"ID":"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6","Type":"ContainerStarted","Data":"4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f"}
Jul 09 04:09:34.197573 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:34.197401    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7" event={"ID":"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6","Type":"ContainerStarted","Data":"5c854e36ae0ef2f7f6e5019b547631b8afc5835938c8d6626cc8fe618f315f84"}
Jul 09 04:09:34.197573 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:34.197487    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:34.213563 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:34.213511    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7" podStartSLOduration=1.793061773 podStartE2EDuration="2.213496339s" totalImagesPullingTime="420.434566ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:09:34.213234728 +0000 UTC m=+1591.032490506" watchObservedRunningTime="2026-07-09 04:09:34.213496339 +0000 UTC m=+1591.032752117"
Jul 09 04:09:36.674552 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:36.674519    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:37.353810 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:37.353774    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:37.440402 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:37.440370    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:37.755376 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:37.755289    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:40.209107 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:40.209073    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:43.416592 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:43.416556    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:43.517019 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:43.516976    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:43.792986 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:43.792945    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:44.148966 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.148859    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"]
Jul 09 04:09:44.149131 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.149094    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7" podUID="fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6" containerName="maas-api" containerID="cri-o://4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f" gracePeriod=30
Jul 09 04:09:44.173589 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.173560    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:44.274641 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.274611    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:44.396892 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.396852    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:44.540968 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.540936    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z477x\" (UniqueName: \"kubernetes.io/projected/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-kube-api-access-z477x\") pod \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") "
Jul 09 04:09:44.541362 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.540987    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-openshift-service-ca\") pod \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") "
Jul 09 04:09:44.541362 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.541011    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-maas-api-tls\") pod \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\" (UID: \"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6\") "
Jul 09 04:09:44.541362 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.541344    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6" (UID: "fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:09:44.543310 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.543287    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-kube-api-access-z477x" (OuterVolumeSpecName: "kube-api-access-z477x") pod "fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6" (UID: "fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6"). InnerVolumeSpecName "kube-api-access-z477x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:09:44.543417 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.543328    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6" (UID: "fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:09:44.641965 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.641929    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-openshift-service-ca\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:09:44.641965 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.641959    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-maas-api-tls\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:09:44.641965 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:44.641970    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z477x\" (UniqueName: \"kubernetes.io/projected/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6-kube-api-access-z477x\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:09:45.251303 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:45.251267    2582 generic.go:356] "Generic (PLEG): container finished" podID="fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6" containerID="4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f" exitCode=0
Jul 09 04:09:45.251490 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:45.251324    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"
Jul 09 04:09:45.251490 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:45.251350    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7" event={"ID":"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6","Type":"ContainerDied","Data":"4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f"}
Jul 09 04:09:45.251490 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:45.251407    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7" event={"ID":"fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6","Type":"ContainerDied","Data":"5c854e36ae0ef2f7f6e5019b547631b8afc5835938c8d6626cc8fe618f315f84"}
Jul 09 04:09:45.251490 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:45.251430    2582 scope.go:117] "RemoveContainer" containerID="4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f"
Jul 09 04:09:45.261722 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:45.261700    2582 scope.go:117] "RemoveContainer" containerID="4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f"
Jul 09 04:09:45.262073 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:09:45.262053    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f\": container with ID starting with 4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f not found: ID does not exist" containerID="4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f"
Jul 09 04:09:45.262151 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:45.262085    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f"} err="failed to get container status \"4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f\": rpc error: code = NotFound desc = could not find container \"4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f\": container with ID starting with 4be5f604414c95c2aa79ccb479c2eb66d958adb2e6f27410955e5f9958fc0a3f not found: ID does not exist"
Jul 09 04:09:45.273090 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:45.273063    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"]
Jul 09 04:09:45.276823 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:45.276795    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99003d6d-79dd7466c-xwpd7"]
Jul 09 04:09:45.713814 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:45.713778    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6" path="/var/lib/kubelet/pods/fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6/volumes"
Jul 09 04:09:50.020923 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:50.020888    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:50.076881 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:50.076819    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:50.365024 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:50.364982    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:50.644449 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:50.644356    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:50.741769 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:50.741712    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:50.847450 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:50.847412    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:56.524294 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:56.524199    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:56.579389 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:56.579349    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:56.654467 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:56.654422    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:56.754732 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:56.754697    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:09:56.897147 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:09:56.897111    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:02.555883 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:02.555831    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:02.656168 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:02.656121    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:09.268751 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:09.268715    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:09.572416 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:09.572373    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:09.913113 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:09.913023    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:12.510308 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.510258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:12.581280 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.581241    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"]
Jul 09 04:10:12.581700 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.581685    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6" containerName="maas-api"
Jul 09 04:10:12.581775 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.581704    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6" containerName="maas-api"
Jul 09 04:10:12.581824 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.581783    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa4ea91b-faf1-4377-ab4c-3b7ee330f2b6" containerName="maas-api"
Jul 09 04:10:12.586342 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.586315    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:12.589530 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.589502    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kmjr\""
Jul 09 04:10:12.589880 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.589845    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a26d209c-serving-cert\""
Jul 09 04:10:12.590905 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.590858    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 04:10:12.591037 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.590911    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 04:10:12.596640 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.596611    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"]
Jul 09 04:10:12.602252 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.602205    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:12.662227 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.662182    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k77fv\" (UniqueName: \"kubernetes.io/projected/db879658-3330-4b9c-9640-2da133cba583-kube-api-access-k77fv\") pod \"maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:12.662411 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.662238    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db879658-3330-4b9c-9640-2da133cba583-maas-api-tls\") pod \"maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:12.662411 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.662288    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db879658-3330-4b9c-9640-2da133cba583-openshift-service-ca\") pod \"maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:12.719065 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.719031    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:12.762849 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.762757    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k77fv\" (UniqueName: \"kubernetes.io/projected/db879658-3330-4b9c-9640-2da133cba583-kube-api-access-k77fv\") pod \"maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:12.762849 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.762809    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db879658-3330-4b9c-9640-2da133cba583-maas-api-tls\") pod \"maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:12.763133 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.762909    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db879658-3330-4b9c-9640-2da133cba583-openshift-service-ca\") pod \"maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:12.763335 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:10:12.763203    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-serving-cert: secret "maas-api-e2e-mt-a26d209c-serving-cert" not found
Jul 09 04:10:12.763335 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:10:12.763299    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db879658-3330-4b9c-9640-2da133cba583-maas-api-tls podName:db879658-3330-4b9c-9640-2da133cba583 nodeName:}" failed. No retries permitted until 2026-07-09 04:10:13.263275964 +0000 UTC m=+1630.082531734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db879658-3330-4b9c-9640-2da133cba583-maas-api-tls") pod "maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj" (UID: "db879658-3330-4b9c-9640-2da133cba583") : secret "maas-api-e2e-mt-a26d209c-serving-cert" not found
Jul 09 04:10:12.763807 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.763781    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db879658-3330-4b9c-9640-2da133cba583-openshift-service-ca\") pod \"maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:12.796538 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.796495    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k77fv\" (UniqueName: \"kubernetes.io/projected/db879658-3330-4b9c-9640-2da133cba583-kube-api-access-k77fv\") pod \"maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:12.867293 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:12.867256    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:13.091181 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:13.091138    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:13.191269 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:13.191230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:13.266378 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:13.266346    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db879658-3330-4b9c-9640-2da133cba583-maas-api-tls\") pod \"maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:13.268905 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:13.268880    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db879658-3330-4b9c-9640-2da133cba583-maas-api-tls\") pod \"maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:13.500139 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:13.500036    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:13.639783 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:13.639757    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"]
Jul 09 04:10:13.642249 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:10:13.642218    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb879658_3330_4b9c_9640_2da133cba583.slice/crio-ee4c2dd5ad3a889b0f6df29dd6dec5e6024f0d69ae2f61d6b64e298ebe0d9a30 WatchSource:0}: Error finding container ee4c2dd5ad3a889b0f6df29dd6dec5e6024f0d69ae2f61d6b64e298ebe0d9a30: Status 404 returned error can't find the container with id ee4c2dd5ad3a889b0f6df29dd6dec5e6024f0d69ae2f61d6b64e298ebe0d9a30
Jul 09 04:10:14.401353 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:14.401315    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj" event={"ID":"db879658-3330-4b9c-9640-2da133cba583","Type":"ContainerStarted","Data":"e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730"}
Jul 09 04:10:14.401538 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:14.401372    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj" event={"ID":"db879658-3330-4b9c-9640-2da133cba583","Type":"ContainerStarted","Data":"ee4c2dd5ad3a889b0f6df29dd6dec5e6024f0d69ae2f61d6b64e298ebe0d9a30"}
Jul 09 04:10:14.401538 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:14.401409    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:14.418596 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:14.418545    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj" podStartSLOduration=2.094459727 podStartE2EDuration="2.41852704s" totalImagesPullingTime="324.067313ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:10:14.418194822 +0000 UTC m=+1631.237450602" watchObservedRunningTime="2026-07-09 04:10:14.41852704 +0000 UTC m=+1631.237782818"
Jul 09 04:10:16.916164 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:16.916127    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:20.412634 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:20.412605    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:10:22.227709 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:22.227667    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:22.328259 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:22.328220    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:25.752372 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:25.752334    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:33.788351 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:33.788315    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:33.889592 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:33.889547    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:49.835403 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.835366    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"]
Jul 09 04:10:49.838533 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.838512    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:49.841887 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.841838    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a26d209c\"/\"openshift-service-ca.crt\""
Jul 09 04:10:49.842029 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.841913    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a26d209c\"/\"kube-root-ca.crt\""
Jul 09 04:10:49.843328 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.843304    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a26d209c\"/\"e2e-lifecycle-model-a26d209c-kserve-self-signed-certs\""
Jul 09 04:10:49.843460 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.843450    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a26d209c\"/\"default-dockercfg-8ldqp\""
Jul 09 04:10:49.848052 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.848030    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"]
Jul 09 04:10:49.861575 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.861544    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:10:49.971552 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.971512    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-model-cache\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:49.971552 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.971556    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tmp-dir\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:49.971785 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.971588    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fw44\" (UniqueName: \"kubernetes.io/projected/f45e09d0-9b3d-4bf1-bd02-73412e95550c-kube-api-access-7fw44\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:49.971785 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.971615    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-dshm\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:49.971785 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.971671    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-home\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:49.971785 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:49.971692    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tls-certs\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.073124 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.073084    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tmp-dir\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.073352 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.073136    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fw44\" (UniqueName: \"kubernetes.io/projected/f45e09d0-9b3d-4bf1-bd02-73412e95550c-kube-api-access-7fw44\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.073352 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.073168    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-dshm\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.073352 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.073221    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-home\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.073352 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.073244    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tls-certs\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.073352 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.073281    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-model-cache\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.073632 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.073556    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tmp-dir\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.073632 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.073597    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-home\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.073720 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.073681    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-model-cache\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.075589 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.075559    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-dshm\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.075840 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.075823    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tls-certs\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.082681 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.082642    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fw44\" (UniqueName: \"kubernetes.io/projected/f45e09d0-9b3d-4bf1-bd02-73412e95550c-kube-api-access-7fw44\") pod \"e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") " pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.149707 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.149609    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:10:50.288191 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.288161    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"]
Jul 09 04:10:50.289424 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:10:50.289383    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf45e09d0_9b3d_4bf1_bd02_73412e95550c.slice/crio-92b91f00ee09d0a1baee2786afaa0bbdbd392e34cc1627ee5f478aa55746fad6 WatchSource:0}: Error finding container 92b91f00ee09d0a1baee2786afaa0bbdbd392e34cc1627ee5f478aa55746fad6: Status 404 returned error can't find the container with id 92b91f00ee09d0a1baee2786afaa0bbdbd392e34cc1627ee5f478aa55746fad6
Jul 09 04:10:50.577382 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.577336    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5" event={"ID":"f45e09d0-9b3d-4bf1-bd02-73412e95550c","Type":"ContainerStarted","Data":"d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636"}
Jul 09 04:10:50.577563 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.577392    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5" event={"ID":"f45e09d0-9b3d-4bf1-bd02-73412e95550c","Type":"ContainerStarted","Data":"92b91f00ee09d0a1baee2786afaa0bbdbd392e34cc1627ee5f478aa55746fad6"}
Jul 09 04:10:50.595321 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:10:50.595261    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5" podStartSLOduration=1.595243803 podStartE2EDuration="1.595243803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:10:50.59258263 +0000 UTC m=+1667.411838411" watchObservedRunningTime="2026-07-09 04:10:50.595243803 +0000 UTC m=+1667.414499604"
Jul 09 04:11:00.150508 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:00.150461    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:11:00.151015 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:00.150544    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:11:00.163419 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:00.163387    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:11:00.642575 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:00.642544    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:11:14.881234 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:14.881181    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:15.207833 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.207744    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"]
Jul 09 04:11:15.208046 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.207996    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj" podUID="db879658-3330-4b9c-9640-2da133cba583" containerName="maas-api" containerID="cri-o://e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730" gracePeriod=30
Jul 09 04:11:15.232168 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.232138    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:15.449285 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.449257    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:11:15.484594 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.484487    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db879658-3330-4b9c-9640-2da133cba583-maas-api-tls\") pod \"db879658-3330-4b9c-9640-2da133cba583\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") "
Jul 09 04:11:15.484594 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.484547    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db879658-3330-4b9c-9640-2da133cba583-openshift-service-ca\") pod \"db879658-3330-4b9c-9640-2da133cba583\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") "
Jul 09 04:11:15.484942 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.484915    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db879658-3330-4b9c-9640-2da133cba583-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db879658-3330-4b9c-9640-2da133cba583" (UID: "db879658-3330-4b9c-9640-2da133cba583"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:11:15.486763 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.486724    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db879658-3330-4b9c-9640-2da133cba583-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db879658-3330-4b9c-9640-2da133cba583" (UID: "db879658-3330-4b9c-9640-2da133cba583"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:11:15.585475 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.585441    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k77fv\" (UniqueName: \"kubernetes.io/projected/db879658-3330-4b9c-9640-2da133cba583-kube-api-access-k77fv\") pod \"db879658-3330-4b9c-9640-2da133cba583\" (UID: \"db879658-3330-4b9c-9640-2da133cba583\") "
Jul 09 04:11:15.585665 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.585591    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db879658-3330-4b9c-9640-2da133cba583-maas-api-tls\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:11:15.585665 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.585603    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db879658-3330-4b9c-9640-2da133cba583-openshift-service-ca\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:11:15.587696 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.587657    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db879658-3330-4b9c-9640-2da133cba583-kube-api-access-k77fv" (OuterVolumeSpecName: "kube-api-access-k77fv") pod "db879658-3330-4b9c-9640-2da133cba583" (UID: "db879658-3330-4b9c-9640-2da133cba583"). InnerVolumeSpecName "kube-api-access-k77fv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:11:15.686833 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.686796    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k77fv\" (UniqueName: \"kubernetes.io/projected/db879658-3330-4b9c-9640-2da133cba583-kube-api-access-k77fv\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:11:15.707135 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.707097    2582 generic.go:356] "Generic (PLEG): container finished" podID="db879658-3330-4b9c-9640-2da133cba583" containerID="e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730" exitCode=0
Jul 09 04:11:15.707309 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.707155    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"
Jul 09 04:11:15.707309 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.707185    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj" event={"ID":"db879658-3330-4b9c-9640-2da133cba583","Type":"ContainerDied","Data":"e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730"}
Jul 09 04:11:15.707309 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.707242    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj" event={"ID":"db879658-3330-4b9c-9640-2da133cba583","Type":"ContainerDied","Data":"ee4c2dd5ad3a889b0f6df29dd6dec5e6024f0d69ae2f61d6b64e298ebe0d9a30"}
Jul 09 04:11:15.707309 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.707266    2582 scope.go:117] "RemoveContainer" containerID="e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730"
Jul 09 04:11:15.716958 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.716937    2582 scope.go:117] "RemoveContainer" containerID="e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730"
Jul 09 04:11:15.717243 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:11:15.717223    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730\": container with ID starting with e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730 not found: ID does not exist" containerID="e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730"
Jul 09 04:11:15.717334 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.717251    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730"} err="failed to get container status \"e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730\": rpc error: code = NotFound desc = could not find container \"e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730\": container with ID starting with e1ac596585f28abde859d2580b3d6854f74c63d1dc504edb0f259ebd039e7730 not found: ID does not exist"
Jul 09 04:11:15.732014 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.731979    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"]
Jul 09 04:11:15.736707 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:15.736600    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj"]
Jul 09 04:11:17.713090 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:17.713056    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db879658-3330-4b9c-9640-2da133cba583" path="/var/lib/kubelet/pods/db879658-3330-4b9c-9640-2da133cba583/volumes"
Jul 09 04:11:18.408227 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:18.408170    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a26d209c-774d47cb7f-9m5jj" podUID="db879658-3330-4b9c-9640-2da133cba583" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.38:8443/health\": context deadline exceeded"
Jul 09 04:11:20.958681 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:20.958649    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:25.943329 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:25.943236    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"]
Jul 09 04:11:25.989459 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:25.989420    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:11:40.963567 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:40.963508    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5" podUID="f45e09d0-9b3d-4bf1-bd02-73412e95550c" containerName="main" containerID="cri-o://d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636" gracePeriod=45
Jul 09 04:11:41.214516 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.214446    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:11:41.315337 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.315299    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-home\") pod \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") "
Jul 09 04:11:41.315535 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.315357    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-model-cache\") pod \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") "
Jul 09 04:11:41.315535 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.315388    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fw44\" (UniqueName: \"kubernetes.io/projected/f45e09d0-9b3d-4bf1-bd02-73412e95550c-kube-api-access-7fw44\") pod \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") "
Jul 09 04:11:41.315535 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.315425    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-dshm\") pod \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") "
Jul 09 04:11:41.315535 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.315461    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tmp-dir\") pod \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") "
Jul 09 04:11:41.315535 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.315494    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tls-certs\") pod \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\" (UID: \"f45e09d0-9b3d-4bf1-bd02-73412e95550c\") "
Jul 09 04:11:41.315818 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.315614    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-home" (OuterVolumeSpecName: "home") pod "f45e09d0-9b3d-4bf1-bd02-73412e95550c" (UID: "f45e09d0-9b3d-4bf1-bd02-73412e95550c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:11:41.315818 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.315700    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-model-cache" (OuterVolumeSpecName: "model-cache") pod "f45e09d0-9b3d-4bf1-bd02-73412e95550c" (UID: "f45e09d0-9b3d-4bf1-bd02-73412e95550c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:11:41.315818 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.315743    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f45e09d0-9b3d-4bf1-bd02-73412e95550c" (UID: "f45e09d0-9b3d-4bf1-bd02-73412e95550c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:11:41.315818 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.315793    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-home\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:11:41.315818 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.315805    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-model-cache\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:11:41.317678 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.317648    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-dshm" (OuterVolumeSpecName: "dshm") pod "f45e09d0-9b3d-4bf1-bd02-73412e95550c" (UID: "f45e09d0-9b3d-4bf1-bd02-73412e95550c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:11:41.317794 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.317689    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f45e09d0-9b3d-4bf1-bd02-73412e95550c" (UID: "f45e09d0-9b3d-4bf1-bd02-73412e95550c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:11:41.317794 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.317771    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f45e09d0-9b3d-4bf1-bd02-73412e95550c-kube-api-access-7fw44" (OuterVolumeSpecName: "kube-api-access-7fw44") pod "f45e09d0-9b3d-4bf1-bd02-73412e95550c" (UID: "f45e09d0-9b3d-4bf1-bd02-73412e95550c"). InnerVolumeSpecName "kube-api-access-7fw44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:11:41.416762 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.416720    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tls-certs\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:11:41.416762 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.416754    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fw44\" (UniqueName: \"kubernetes.io/projected/f45e09d0-9b3d-4bf1-bd02-73412e95550c-kube-api-access-7fw44\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:11:41.416762 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.416767    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-dshm\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:11:41.417043 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.416779    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f45e09d0-9b3d-4bf1-bd02-73412e95550c-tmp-dir\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:11:41.830673 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.830634    2582 generic.go:356] "Generic (PLEG): container finished" podID="f45e09d0-9b3d-4bf1-bd02-73412e95550c" containerID="d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636" exitCode=0
Jul 09 04:11:41.830673 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.830675    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"
Jul 09 04:11:41.830945 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.830686    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5" event={"ID":"f45e09d0-9b3d-4bf1-bd02-73412e95550c","Type":"ContainerDied","Data":"d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636"}
Jul 09 04:11:41.830945 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.830728    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5" event={"ID":"f45e09d0-9b3d-4bf1-bd02-73412e95550c","Type":"ContainerDied","Data":"92b91f00ee09d0a1baee2786afaa0bbdbd392e34cc1627ee5f478aa55746fad6"}
Jul 09 04:11:41.830945 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.830746    2582 scope.go:117] "RemoveContainer" containerID="d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636"
Jul 09 04:11:41.839309 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.839290    2582 scope.go:117] "RemoveContainer" containerID="d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636"
Jul 09 04:11:41.839606 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:11:41.839586    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636\": container with ID starting with d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636 not found: ID does not exist" containerID="d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636"
Jul 09 04:11:41.839687 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.839619    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636"} err="failed to get container status \"d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636\": rpc error: code = NotFound desc = could not find container \"d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636\": container with ID starting with d4deca41baf9f155dc372b489e9dc19d9462e118aeb03908b826913c09dcf636 not found: ID does not exist"
Jul 09 04:11:41.848421 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.848392    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"]
Jul 09 04:11:41.852964 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:41.852834    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-a26d209c/e2e-lifecycle-model-a26d209c-kserve-c7cc488d7-rnrl5"]
Jul 09 04:11:43.714881 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:11:43.714826    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f45e09d0-9b3d-4bf1-bd02-73412e95550c" path="/var/lib/kubelet/pods/f45e09d0-9b3d-4bf1-bd02-73412e95550c/volumes"
Jul 09 04:12:02.388556 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:02.388512    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:04.543584 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:04.543548    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:08.705825 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:08.705778    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:08.767412 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:08.767369    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:08.989304 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:08.989181    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:10.227008 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:10.226970    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:10.328081 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:10.328030    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:10.421562 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:10.421527    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:11.148094 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:11.148063    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:11.238704 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:11.238669    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:11.339221 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:11.339184    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:12.691749 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:12.691710    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:18.410764 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:18.410728    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:18.473523 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:18.473484    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:19.072394 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:19.072354    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:24.861052 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:24.861003    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:24.963834 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:24.963802    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:25.061432 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:25.061394    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:35.577741 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:35.577695    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:35.900634 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:35.900540    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:12:56.822642 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:12:56.822550    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:02.566885 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:02.566838    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:02.626525 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:02.626478    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:02.903093 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:02.902993    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:03.216498 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:03.216404    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:03.714963 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:03.714931    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 04:13:03.717731 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:03.717706    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 04:13:03.717961 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:03.717947    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 04:13:03.720444 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:03.720421    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 04:13:03.931564 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:03.931519    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:06.344017 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.343974    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:06.432408 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.432368    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:06.435370 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.435340    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"]
Jul 09 04:13:06.435706 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.435690    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db879658-3330-4b9c-9640-2da133cba583" containerName="maas-api"
Jul 09 04:13:06.435789 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.435710    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="db879658-3330-4b9c-9640-2da133cba583" containerName="maas-api"
Jul 09 04:13:06.435789 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.435724    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f45e09d0-9b3d-4bf1-bd02-73412e95550c" containerName="main"
Jul 09 04:13:06.435789 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.435733    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f45e09d0-9b3d-4bf1-bd02-73412e95550c" containerName="main"
Jul 09 04:13:06.435963 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.435807    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f45e09d0-9b3d-4bf1-bd02-73412e95550c" containerName="main"
Jul 09 04:13:06.435963 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.435823    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="db879658-3330-4b9c-9640-2da133cba583" containerName="maas-api"
Jul 09 04:13:06.438763 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.438743    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:06.442941 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.442916    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2eb6ca-serving-cert\""
Jul 09 04:13:06.443063 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.442939    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kmjr\""
Jul 09 04:13:06.443779 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.443765    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 04:13:06.444236 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.444219    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 04:13:06.450810 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.450779    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"]
Jul 09 04:13:06.538408 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.538366    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:06.550694 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.550655    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/535d8eef-698d-4390-a8b3-433fdfc4cfc0-maas-api-tls\") pod \"maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:06.550909 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.550771    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7v6w\" (UniqueName: \"kubernetes.io/projected/535d8eef-698d-4390-a8b3-433fdfc4cfc0-kube-api-access-x7v6w\") pod \"maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:06.550909 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.550823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/535d8eef-698d-4390-a8b3-433fdfc4cfc0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:06.652391 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.652291    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7v6w\" (UniqueName: \"kubernetes.io/projected/535d8eef-698d-4390-a8b3-433fdfc4cfc0-kube-api-access-x7v6w\") pod \"maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:06.652391 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.652343    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/535d8eef-698d-4390-a8b3-433fdfc4cfc0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:06.652391 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.652374    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/535d8eef-698d-4390-a8b3-433fdfc4cfc0-maas-api-tls\") pod \"maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:06.652703 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:13:06.652492    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-serving-cert: secret "maas-api-e2e-inf-a-2eb6ca-serving-cert" not found
Jul 09 04:13:06.652703 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:13:06.652555    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/535d8eef-698d-4390-a8b3-433fdfc4cfc0-maas-api-tls podName:535d8eef-698d-4390-a8b3-433fdfc4cfc0 nodeName:}" failed. No retries permitted until 2026-07-09 04:13:07.152539134 +0000 UTC m=+1803.971794890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/535d8eef-698d-4390-a8b3-433fdfc4cfc0-maas-api-tls") pod "maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx" (UID: "535d8eef-698d-4390-a8b3-433fdfc4cfc0") : secret "maas-api-e2e-inf-a-2eb6ca-serving-cert" not found
Jul 09 04:13:06.653071 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.653051    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/535d8eef-698d-4390-a8b3-433fdfc4cfc0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:06.662117 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:06.662083    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7v6w\" (UniqueName: \"kubernetes.io/projected/535d8eef-698d-4390-a8b3-433fdfc4cfc0-kube-api-access-x7v6w\") pod \"maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:07.049003 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:07.048971    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:07.148967 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:07.148930    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:07.157079 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:07.157050    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/535d8eef-698d-4390-a8b3-433fdfc4cfc0-maas-api-tls\") pod \"maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:07.159598 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:07.159579    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/535d8eef-698d-4390-a8b3-433fdfc4cfc0-maas-api-tls\") pod \"maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:07.251012 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:07.250972    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:07.350285 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:07.350186    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:07.488763 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:07.488731    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"]
Jul 09 04:13:07.491174 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:13:07.491147    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod535d8eef_698d_4390_a8b3_433fdfc4cfc0.slice/crio-e3975b8a511d80aa3288c72e61b510f56d0fbdd63b7578a1dc2cde5649655cb8 WatchSource:0}: Error finding container e3975b8a511d80aa3288c72e61b510f56d0fbdd63b7578a1dc2cde5649655cb8: Status 404 returned error can't find the container with id e3975b8a511d80aa3288c72e61b510f56d0fbdd63b7578a1dc2cde5649655cb8
Jul 09 04:13:07.492905 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:07.492885    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 04:13:08.249746 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:08.249710    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx" event={"ID":"535d8eef-698d-4390-a8b3-433fdfc4cfc0","Type":"ContainerStarted","Data":"535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36"}
Jul 09 04:13:08.249978 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:08.249766    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx" event={"ID":"535d8eef-698d-4390-a8b3-433fdfc4cfc0","Type":"ContainerStarted","Data":"e3975b8a511d80aa3288c72e61b510f56d0fbdd63b7578a1dc2cde5649655cb8"}
Jul 09 04:13:08.249978 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:08.249817    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:08.269903 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:08.269829    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx" podStartSLOduration=1.650872257 podStartE2EDuration="2.269808269s" totalImagesPullingTime="618.936012ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:13:08.268085452 +0000 UTC m=+1805.087341229" watchObservedRunningTime="2026-07-09 04:13:08.269808269 +0000 UTC m=+1805.089064048"
Jul 09 04:13:10.829661 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:10.829618    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:11.160520 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:11.160426    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:11.447202 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:11.447095    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:11.657106 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:11.657069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:14.262508 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.262478    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:13:14.459826 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.459787    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:14.531588 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.531556    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"]
Jul 09 04:13:14.533542 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.533499    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:14.535188 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.535168    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:14.537788 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.537768    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-2dbf5f-serving-cert\""
Jul 09 04:13:14.545696 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.545669    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"]
Jul 09 04:13:14.591328 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.591285    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:14.619365 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.619327    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfjjk\" (UniqueName: \"kubernetes.io/projected/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-kube-api-access-bfjjk\") pod \"maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:14.619525 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.619378    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:14.619525 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.619482    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:14.720238 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.720191    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfjjk\" (UniqueName: \"kubernetes.io/projected/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-kube-api-access-bfjjk\") pod \"maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:14.720238 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.720246    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:14.720523 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.720284    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:14.720523 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:13:14.720423    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-serving-cert: secret "maas-api-e2e-inf-b-2dbf5f-serving-cert" not found
Jul 09 04:13:14.720523 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:13:14.720514    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-maas-api-tls podName:2771aab7-0bb6-4f82-823b-7992f3ba8a4f nodeName:}" failed. No retries permitted until 2026-07-09 04:13:15.22049246 +0000 UTC m=+1812.039748232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-maas-api-tls") pod "maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql" (UID: "2771aab7-0bb6-4f82-823b-7992f3ba8a4f") : secret "maas-api-e2e-inf-b-2dbf5f-serving-cert" not found
Jul 09 04:13:14.720939 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.720919    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:14.732940 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.732904    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfjjk\" (UniqueName: \"kubernetes.io/projected/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-kube-api-access-bfjjk\") pod \"maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:14.778602 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:14.778565    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:15.002097 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:15.001985    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:15.101729 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:15.101691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:15.225390 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:15.225354    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:15.227908 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:15.227883    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:15.446755 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:15.446709    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:15.588647 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:15.588617    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"]
Jul 09 04:13:15.590904 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:13:15.590859    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2771aab7_0bb6_4f82_823b_7992f3ba8a4f.slice/crio-af162d0add242e8db9799f0acd53b6a939ac9806d6fbfb0084a0764606d79dad WatchSource:0}: Error finding container af162d0add242e8db9799f0acd53b6a939ac9806d6fbfb0084a0764606d79dad: Status 404 returned error can't find the container with id af162d0add242e8db9799f0acd53b6a939ac9806d6fbfb0084a0764606d79dad
Jul 09 04:13:16.299495 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:16.299454    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql" event={"ID":"2771aab7-0bb6-4f82-823b-7992f3ba8a4f","Type":"ContainerStarted","Data":"5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b"}
Jul 09 04:13:16.299699 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:16.299515    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql" event={"ID":"2771aab7-0bb6-4f82-823b-7992f3ba8a4f","Type":"ContainerStarted","Data":"af162d0add242e8db9799f0acd53b6a939ac9806d6fbfb0084a0764606d79dad"}
Jul 09 04:13:16.299699 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:16.299536    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:16.318006 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:16.317945    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql" podStartSLOduration=1.92822391 podStartE2EDuration="2.317926688s" totalImagesPullingTime="389.702778ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:13:16.316991147 +0000 UTC m=+1813.136246926" watchObservedRunningTime="2026-07-09 04:13:16.317926688 +0000 UTC m=+1813.137182490"
Jul 09 04:13:18.872471 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:18.872434    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:22.309981 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:22.309947    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:13:28.678143 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:28.678097    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:28.779121 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:28.779087    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:29.251515 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:29.251481    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:35.799990 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:35.799952    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:35.899798 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:35.899756    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:13:51.463012 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:13:51.462967    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:12.236357 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:12.236316    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:13.137422 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:13.137385    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:13.302249 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:13.302211    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:13.694194 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:13.694158    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:13.795217 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:13.795168    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:13.858581 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:13.858546    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:13.930431 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:13.930395    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"]
Jul 09 04:14:13.930944 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:13.930908    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx" podUID="535d8eef-698d-4390-a8b3-433fdfc4cfc0" containerName="maas-api" containerID="cri-o://535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36" gracePeriod=30
Jul 09 04:14:13.957331 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:13.957227    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:14.256899 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.256845    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx" podUID="535d8eef-698d-4390-a8b3-433fdfc4cfc0" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.40:8443/health\": dial tcp 10.134.0.40:8443: connect: connection refused"
Jul 09 04:14:14.378161 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.378133    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:14:14.446895 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.446831    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7v6w\" (UniqueName: \"kubernetes.io/projected/535d8eef-698d-4390-a8b3-433fdfc4cfc0-kube-api-access-x7v6w\") pod \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") "
Jul 09 04:14:14.447116 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.446932    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/535d8eef-698d-4390-a8b3-433fdfc4cfc0-openshift-service-ca\") pod \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") "
Jul 09 04:14:14.447116 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.446988    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/535d8eef-698d-4390-a8b3-433fdfc4cfc0-maas-api-tls\") pod \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\" (UID: \"535d8eef-698d-4390-a8b3-433fdfc4cfc0\") "
Jul 09 04:14:14.447350 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.447325    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/535d8eef-698d-4390-a8b3-433fdfc4cfc0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "535d8eef-698d-4390-a8b3-433fdfc4cfc0" (UID: "535d8eef-698d-4390-a8b3-433fdfc4cfc0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:14:14.449066 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.449041    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/535d8eef-698d-4390-a8b3-433fdfc4cfc0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "535d8eef-698d-4390-a8b3-433fdfc4cfc0" (UID: "535d8eef-698d-4390-a8b3-433fdfc4cfc0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:14:14.449160 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.449132    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/535d8eef-698d-4390-a8b3-433fdfc4cfc0-kube-api-access-x7v6w" (OuterVolumeSpecName: "kube-api-access-x7v6w") pod "535d8eef-698d-4390-a8b3-433fdfc4cfc0" (UID: "535d8eef-698d-4390-a8b3-433fdfc4cfc0"). InnerVolumeSpecName "kube-api-access-x7v6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:14:14.547568 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.547536    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/535d8eef-698d-4390-a8b3-433fdfc4cfc0-maas-api-tls\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:14:14.547568 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.547571    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7v6w\" (UniqueName: \"kubernetes.io/projected/535d8eef-698d-4390-a8b3-433fdfc4cfc0-kube-api-access-x7v6w\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:14:14.547789 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.547583    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/535d8eef-698d-4390-a8b3-433fdfc4cfc0-openshift-service-ca\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:14:14.583723 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.583678    2582 generic.go:356] "Generic (PLEG): container finished" podID="535d8eef-698d-4390-a8b3-433fdfc4cfc0" containerID="535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36" exitCode=0
Jul 09 04:14:14.583931 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.583740    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"
Jul 09 04:14:14.583931 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.583756    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx" event={"ID":"535d8eef-698d-4390-a8b3-433fdfc4cfc0","Type":"ContainerDied","Data":"535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36"}
Jul 09 04:14:14.583931 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.583818    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx" event={"ID":"535d8eef-698d-4390-a8b3-433fdfc4cfc0","Type":"ContainerDied","Data":"e3975b8a511d80aa3288c72e61b510f56d0fbdd63b7578a1dc2cde5649655cb8"}
Jul 09 04:14:14.583931 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.583842    2582 scope.go:117] "RemoveContainer" containerID="535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36"
Jul 09 04:14:14.593287 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.593259    2582 scope.go:117] "RemoveContainer" containerID="535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36"
Jul 09 04:14:14.593741 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:14:14.593593    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36\": container with ID starting with 535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36 not found: ID does not exist" containerID="535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36"
Jul 09 04:14:14.593741 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.593628    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36"} err="failed to get container status \"535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36\": rpc error: code = NotFound desc = could not find container \"535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36\": container with ID starting with 535c0bebce0e4cf1573115241bb44e8b5ac511b4c832898733e27b60d47cdb36 not found: ID does not exist"
Jul 09 04:14:14.608963 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.608923    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"]
Jul 09 04:14:14.614025 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:14.613992    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2eb6ca-f59686fd5-w45kx"]
Jul 09 04:14:15.713523 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:15.713481    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="535d8eef-698d-4390-a8b3-433fdfc4cfc0" path="/var/lib/kubelet/pods/535d8eef-698d-4390-a8b3-433fdfc4cfc0/volumes"
Jul 09 04:14:18.763460 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:18.763411    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:55.312908 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:55.312846    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:55.586624 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:55.586518    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:56.134447 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:56.134415    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:56.501572 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:56.501460    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:56.602603 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:56.602561    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:56.703019 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:56.702985    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:56.764618 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:56.764520    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"]
Jul 09 04:14:56.764789 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:56.764760    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql" podUID="2771aab7-0bb6-4f82-823b-7992f3ba8a4f" containerName="maas-api" containerID="cri-o://5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b" gracePeriod=30
Jul 09 04:14:56.796856 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:56.796824    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:14:57.014536 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.014509    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:14:57.111971 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.111922    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfjjk\" (UniqueName: \"kubernetes.io/projected/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-kube-api-access-bfjjk\") pod \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") "
Jul 09 04:14:57.111971 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.111965    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-maas-api-tls\") pod \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") "
Jul 09 04:14:57.112269 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.112009    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-openshift-service-ca\") pod \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\" (UID: \"2771aab7-0bb6-4f82-823b-7992f3ba8a4f\") "
Jul 09 04:14:57.112429 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.112396    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2771aab7-0bb6-4f82-823b-7992f3ba8a4f" (UID: "2771aab7-0bb6-4f82-823b-7992f3ba8a4f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:14:57.114178 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.114154    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-kube-api-access-bfjjk" (OuterVolumeSpecName: "kube-api-access-bfjjk") pod "2771aab7-0bb6-4f82-823b-7992f3ba8a4f" (UID: "2771aab7-0bb6-4f82-823b-7992f3ba8a4f"). InnerVolumeSpecName "kube-api-access-bfjjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:14:57.114318 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.114297    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2771aab7-0bb6-4f82-823b-7992f3ba8a4f" (UID: "2771aab7-0bb6-4f82-823b-7992f3ba8a4f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:14:57.213510 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.213471    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfjjk\" (UniqueName: \"kubernetes.io/projected/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-kube-api-access-bfjjk\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:14:57.213510 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.213504    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-maas-api-tls\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:14:57.213510 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.213513    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2771aab7-0bb6-4f82-823b-7992f3ba8a4f-openshift-service-ca\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:14:57.793595 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.793562    2582 generic.go:356] "Generic (PLEG): container finished" podID="2771aab7-0bb6-4f82-823b-7992f3ba8a4f" containerID="5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b" exitCode=0
Jul 09 04:14:57.794069 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.793624    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"
Jul 09 04:14:57.794069 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.793636    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql" event={"ID":"2771aab7-0bb6-4f82-823b-7992f3ba8a4f","Type":"ContainerDied","Data":"5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b"}
Jul 09 04:14:57.794069 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.793691    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql" event={"ID":"2771aab7-0bb6-4f82-823b-7992f3ba8a4f","Type":"ContainerDied","Data":"af162d0add242e8db9799f0acd53b6a939ac9806d6fbfb0084a0764606d79dad"}
Jul 09 04:14:57.794069 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.793717    2582 scope.go:117] "RemoveContainer" containerID="5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b"
Jul 09 04:14:57.802335 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.802307    2582 scope.go:117] "RemoveContainer" containerID="5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b"
Jul 09 04:14:57.802606 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:14:57.802585    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b\": container with ID starting with 5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b not found: ID does not exist" containerID="5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b"
Jul 09 04:14:57.802703 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.802615    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b"} err="failed to get container status \"5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b\": rpc error: code = NotFound desc = could not find container \"5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b\": container with ID starting with 5d7f4ea861d590768940be4800ea46da04360227e850d69afc50af011929697b not found: ID does not exist"
Jul 09 04:14:57.812317 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.812275    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"]
Jul 09 04:14:57.816255 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:57.816227    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbf5f-5fcf47664-mccql"]
Jul 09 04:14:59.717766 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:14:59.717727    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2771aab7-0bb6-4f82-823b-7992f3ba8a4f" path="/var/lib/kubelet/pods/2771aab7-0bb6-4f82-823b-7992f3ba8a4f/volumes"
Jul 09 04:15:00.163841 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.163799    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"]
Jul 09 04:15:00.164217 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.164197    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="535d8eef-698d-4390-a8b3-433fdfc4cfc0" containerName="maas-api"
Jul 09 04:15:00.164308 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.164221    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="535d8eef-698d-4390-a8b3-433fdfc4cfc0" containerName="maas-api"
Jul 09 04:15:00.164308 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.164250    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2771aab7-0bb6-4f82-823b-7992f3ba8a4f" containerName="maas-api"
Jul 09 04:15:00.164308 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.164259    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2771aab7-0bb6-4f82-823b-7992f3ba8a4f" containerName="maas-api"
Jul 09 04:15:00.164462 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.164347    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2771aab7-0bb6-4f82-823b-7992f3ba8a4f" containerName="maas-api"
Jul 09 04:15:00.164462 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.164363    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="535d8eef-698d-4390-a8b3-433fdfc4cfc0" containerName="maas-api"
Jul 09 04:15:00.168841 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.168815    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"
Jul 09 04:15:00.172775 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.172748    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kmjr\""
Jul 09 04:15:00.172943 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.172878    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 04:15:00.174142 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.174126    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 04:15:00.193021 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.192987    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"]
Jul 09 04:15:00.241579 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.241538    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd8sv\" (UniqueName: \"kubernetes.io/projected/042df7af-eaf9-435b-9c72-75474ab8029d-kube-api-access-hd8sv\") pod \"maas-api-key-cleanup-29726175-jvqtx\" (UID: \"042df7af-eaf9-435b-9c72-75474ab8029d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"
Jul 09 04:15:00.342242 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.342204    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd8sv\" (UniqueName: \"kubernetes.io/projected/042df7af-eaf9-435b-9c72-75474ab8029d-kube-api-access-hd8sv\") pod \"maas-api-key-cleanup-29726175-jvqtx\" (UID: \"042df7af-eaf9-435b-9c72-75474ab8029d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"
Jul 09 04:15:00.352116 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.352078    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd8sv\" (UniqueName: \"kubernetes.io/projected/042df7af-eaf9-435b-9c72-75474ab8029d-kube-api-access-hd8sv\") pod \"maas-api-key-cleanup-29726175-jvqtx\" (UID: \"042df7af-eaf9-435b-9c72-75474ab8029d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"
Jul 09 04:15:00.479171 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.479077    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"
Jul 09 04:15:00.618942 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.618910    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"]
Jul 09 04:15:00.621499 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:15:00.621463    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod042df7af_eaf9_435b_9c72_75474ab8029d.slice/crio-2f409f6bdb7bfa8962fd89a7d965ee265d1eb7cbd5030b78a7bf4d2138ef0c5e WatchSource:0}: Error finding container 2f409f6bdb7bfa8962fd89a7d965ee265d1eb7cbd5030b78a7bf4d2138ef0c5e: Status 404 returned error can't find the container with id 2f409f6bdb7bfa8962fd89a7d965ee265d1eb7cbd5030b78a7bf4d2138ef0c5e
Jul 09 04:15:00.813519 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:00.813483    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx" event={"ID":"042df7af-eaf9-435b-9c72-75474ab8029d","Type":"ContainerStarted","Data":"2f409f6bdb7bfa8962fd89a7d965ee265d1eb7cbd5030b78a7bf4d2138ef0c5e"}
Jul 09 04:15:01.819306 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:01.819270    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx" event={"ID":"042df7af-eaf9-435b-9c72-75474ab8029d","Type":"ContainerStarted","Data":"95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4"}
Jul 09 04:15:01.836458 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:01.836404    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx" podStartSLOduration=1.836384179 podStartE2EDuration="1.836384179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:01.834405496 +0000 UTC m=+1918.653661275" watchObservedRunningTime="2026-07-09 04:15:01.836384179 +0000 UTC m=+1918.655639956"
Jul 09 04:15:02.454587 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:02.454553    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:02.645258 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:02.645221    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:02.740677 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:02.740584    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:03.318746 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.318704    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:03.657179 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.657083    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"]
Jul 09 04:15:03.660470 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.660452    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.664080 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.664056    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c44f7d-openshift-default-dockercfg-mrtwh\""
Jul 09 04:15:03.675065 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.675030    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"]
Jul 09 04:15:03.686922 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.686860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:03.773341 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.773309    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-podinfo\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.773521 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.773356    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-token\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.773521 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.773452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-credential-socket\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.773521 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.773494    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-envoy\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.773521 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.773520    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa99af78-e69d-4afb-b55c-95e3573282b8-istiod-ca-cert\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.773673 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.773610    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-data\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.773673 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.773647    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-certs\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.773673 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.773669    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfncm\" (UniqueName: \"kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-kube-api-access-rfncm\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.773786 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.773721    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-socket\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.792179 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.792137    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:03.874705 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.874648    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-credential-socket\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.874705 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.874707    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-envoy\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875001 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.874817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa99af78-e69d-4afb-b55c-95e3573282b8-istiod-ca-cert\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875001 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.874888    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-data\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875001 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.874916    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-certs\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875001 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.874938    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfncm\" (UniqueName: \"kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-kube-api-access-rfncm\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875001 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.874961    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-socket\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875241 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.875012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-podinfo\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875241 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.875073    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-token\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875241 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.875172    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-credential-socket\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875381 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.875332    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-certs\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875543 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.875505    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-socket\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875614 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.875586    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa99af78-e69d-4afb-b55c-95e3573282b8-istiod-ca-cert\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.875614 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.875597    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-data\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.877395 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.877335    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-envoy\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.877735 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.877714    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-podinfo\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.883964 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.883913    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-token\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.884412 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.884394    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfncm\" (UniqueName: \"kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-kube-api-access-rfncm\") pod \"e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") " pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:03.970585 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:03.970490    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:04.118668 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:04.118634    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"]
Jul 09 04:15:04.119757 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:15:04.119728    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa99af78_e69d_4afb_b55c_95e3573282b8.slice/crio-8467fad1d2ac8ca50ebe22bc640953b07cc9237421757a7e2fb89ffe4f02d876 WatchSource:0}: Error finding container 8467fad1d2ac8ca50ebe22bc640953b07cc9237421757a7e2fb89ffe4f02d876: Status 404 returned error can't find the container with id 8467fad1d2ac8ca50ebe22bc640953b07cc9237421757a7e2fb89ffe4f02d876
Jul 09 04:15:04.837538 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:04.837500    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" event={"ID":"fa99af78-e69d-4afb-b55c-95e3573282b8","Type":"ContainerStarted","Data":"8467fad1d2ac8ca50ebe22bc640953b07cc9237421757a7e2fb89ffe4f02d876"}
Jul 09 04:15:08.453703 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:08.453667    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:15:08.453951 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:08.453743    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:15:08.453951 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:08.453774    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:15:08.867745 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:08.867709    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" event={"ID":"fa99af78-e69d-4afb-b55c-95e3573282b8","Type":"ContainerStarted","Data":"7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b"}
Jul 09 04:15:08.890894 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:08.890818    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" podStartSLOduration=1.5593658929999998 podStartE2EDuration="5.8908002s" totalImagesPullingTime="4.331434307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:08.888773584 +0000 UTC m=+1925.708029363" watchObservedRunningTime="2026-07-09 04:15:08.8908002 +0000 UTC m=+1925.710055978"
Jul 09 04:15:08.971695 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:08.971655    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:09.976651 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:09.976620    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:10.826590 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:10.826552    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:10.876800 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:10.876763    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:10.877833 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:10.877814    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:10.955683 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:10.955647    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"]
Jul 09 04:15:10.959507 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:10.959481    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:10.963010 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:10.962984    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c44f7d-serving-cert\""
Jul 09 04:15:10.989536 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:10.989468    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:10.991173 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:10.991149    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"]
Jul 09 04:15:11.041614 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.041571    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49tnt\" (UniqueName: \"kubernetes.io/projected/a28a97d9-edfd-40b1-bef6-afaa03691eb2-kube-api-access-49tnt\") pod \"maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:11.041834 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.041622    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a28a97d9-edfd-40b1-bef6-afaa03691eb2-maas-api-tls\") pod \"maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:11.041834 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.041816    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28a97d9-edfd-40b1-bef6-afaa03691eb2-openshift-service-ca\") pod \"maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:11.102229 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.102135    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:11.143268 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.143235    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28a97d9-edfd-40b1-bef6-afaa03691eb2-openshift-service-ca\") pod \"maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:11.143489 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.143315    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49tnt\" (UniqueName: \"kubernetes.io/projected/a28a97d9-edfd-40b1-bef6-afaa03691eb2-kube-api-access-49tnt\") pod \"maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:11.143489 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.143339    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a28a97d9-edfd-40b1-bef6-afaa03691eb2-maas-api-tls\") pod \"maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:11.144043 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.144010    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28a97d9-edfd-40b1-bef6-afaa03691eb2-openshift-service-ca\") pod \"maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:11.146068 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.146047    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a28a97d9-edfd-40b1-bef6-afaa03691eb2-maas-api-tls\") pod \"maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:11.156128 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.156100    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49tnt\" (UniqueName: \"kubernetes.io/projected/a28a97d9-edfd-40b1-bef6-afaa03691eb2-kube-api-access-49tnt\") pod \"maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:11.198844 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.198804    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:11.271246 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.271205    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:11.413941 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.413915    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"]
Jul 09 04:15:11.415274 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:15:11.415232    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda28a97d9_edfd_40b1_bef6_afaa03691eb2.slice/crio-ccae14ab97f5665d924ffcf762fdf7664bd46daf73076d982503df66291cfffe WatchSource:0}: Error finding container ccae14ab97f5665d924ffcf762fdf7664bd46daf73076d982503df66291cfffe: Status 404 returned error can't find the container with id ccae14ab97f5665d924ffcf762fdf7664bd46daf73076d982503df66291cfffe
Jul 09 04:15:11.502309 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.502272    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:11.602233 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.602195    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:11.885424 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:11.885383    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj" event={"ID":"a28a97d9-edfd-40b1-bef6-afaa03691eb2","Type":"ContainerStarted","Data":"ccae14ab97f5665d924ffcf762fdf7664bd46daf73076d982503df66291cfffe"}
Jul 09 04:15:12.891676 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:12.891638    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj" event={"ID":"a28a97d9-edfd-40b1-bef6-afaa03691eb2","Type":"ContainerStarted","Data":"768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9"}
Jul 09 04:15:12.892158 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:12.891892    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:12.909120 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:12.909066    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj" podStartSLOduration=2.33493075 podStartE2EDuration="2.909048841s" totalImagesPullingTime="574.118091ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:12.908512144 +0000 UTC m=+1929.727767947" watchObservedRunningTime="2026-07-09 04:15:12.909048841 +0000 UTC m=+1929.728304619"
Jul 09 04:15:15.339993 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:15.339959    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:15.810170 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:15.810136    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:16.114102 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:16.113989    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:16.342616 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:16.342581    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:18.907054 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:18.907020    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:19.106891 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.106583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:19.260318 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.260232    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:19.284477 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.284423    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"]
Jul 09 04:15:19.288403 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.288372    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:19.291980 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.291953    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f007b3-serving-cert\""
Jul 09 04:15:19.315179 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.315138    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"]
Jul 09 04:15:19.370746 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.370710    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:19.423414 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.423378    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgnvl\" (UniqueName: \"kubernetes.io/projected/81adc359-e062-49ca-aa92-4e49eb680056-kube-api-access-hgnvl\") pod \"maas-api-e2e-api-b-f007b3-5859884c44-958h5\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:19.423575 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.423503    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81adc359-e062-49ca-aa92-4e49eb680056-openshift-service-ca\") pod \"maas-api-e2e-api-b-f007b3-5859884c44-958h5\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:19.423575 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.423537    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81adc359-e062-49ca-aa92-4e49eb680056-maas-api-tls\") pod \"maas-api-e2e-api-b-f007b3-5859884c44-958h5\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:19.524286 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.524203    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81adc359-e062-49ca-aa92-4e49eb680056-openshift-service-ca\") pod \"maas-api-e2e-api-b-f007b3-5859884c44-958h5\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:19.524286 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.524240    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81adc359-e062-49ca-aa92-4e49eb680056-maas-api-tls\") pod \"maas-api-e2e-api-b-f007b3-5859884c44-958h5\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:19.524286 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.524276    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgnvl\" (UniqueName: \"kubernetes.io/projected/81adc359-e062-49ca-aa92-4e49eb680056-kube-api-access-hgnvl\") pod \"maas-api-e2e-api-b-f007b3-5859884c44-958h5\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:19.524535 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:15:19.524402    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-serving-cert: secret "maas-api-e2e-api-b-f007b3-serving-cert" not found
Jul 09 04:15:19.524535 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:15:19.524461    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81adc359-e062-49ca-aa92-4e49eb680056-maas-api-tls podName:81adc359-e062-49ca-aa92-4e49eb680056 nodeName:}" failed. No retries permitted until 2026-07-09 04:15:20.024443831 +0000 UTC m=+1936.843699587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81adc359-e062-49ca-aa92-4e49eb680056-maas-api-tls") pod "maas-api-e2e-api-b-f007b3-5859884c44-958h5" (UID: "81adc359-e062-49ca-aa92-4e49eb680056") : secret "maas-api-e2e-api-b-f007b3-serving-cert" not found
Jul 09 04:15:19.524828 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.524807    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81adc359-e062-49ca-aa92-4e49eb680056-openshift-service-ca\") pod \"maas-api-e2e-api-b-f007b3-5859884c44-958h5\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:19.534010 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.533977    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgnvl\" (UniqueName: \"kubernetes.io/projected/81adc359-e062-49ca-aa92-4e49eb680056-kube-api-access-hgnvl\") pod \"maas-api-e2e-api-b-f007b3-5859884c44-958h5\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:19.678662 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.678626    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:19.826309 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.826258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:19.927486 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:19.927435    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:20.028155 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:20.028056    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81adc359-e062-49ca-aa92-4e49eb680056-maas-api-tls\") pod \"maas-api-e2e-api-b-f007b3-5859884c44-958h5\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:20.030725 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:20.030693    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81adc359-e062-49ca-aa92-4e49eb680056-maas-api-tls\") pod \"maas-api-e2e-api-b-f007b3-5859884c44-958h5\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:20.204536 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:20.204489    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:20.363344 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:20.363309    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"]
Jul 09 04:15:20.367128 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:15:20.367099    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81adc359_e062_49ca_aa92_4e49eb680056.slice/crio-a92aa77d3604db47520e5d5279ffc9fd10abfa7d2dbb37ecb536467efffe9d9d WatchSource:0}: Error finding container a92aa77d3604db47520e5d5279ffc9fd10abfa7d2dbb37ecb536467efffe9d9d: Status 404 returned error can't find the container with id a92aa77d3604db47520e5d5279ffc9fd10abfa7d2dbb37ecb536467efffe9d9d
Jul 09 04:15:20.939458 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:20.939413    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5" event={"ID":"81adc359-e062-49ca-aa92-4e49eb680056","Type":"ContainerStarted","Data":"36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f"}
Jul 09 04:15:20.939856 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:20.939480    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:20.939856 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:20.939496    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5" event={"ID":"81adc359-e062-49ca-aa92-4e49eb680056","Type":"ContainerStarted","Data":"a92aa77d3604db47520e5d5279ffc9fd10abfa7d2dbb37ecb536467efffe9d9d"}
Jul 09 04:15:20.958210 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:20.958146    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5" podStartSLOduration=1.556849844 podStartE2EDuration="1.958129227s" totalImagesPullingTime="401.279383ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:20.955784215 +0000 UTC m=+1937.775040009" watchObservedRunningTime="2026-07-09 04:15:20.958129227 +0000 UTC m=+1937.777385060"
Jul 09 04:15:22.934260 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:22.934151    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:22.951636 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:22.951605    2582 generic.go:356] "Generic (PLEG): container finished" podID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerID="7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b" exitCode=0
Jul 09 04:15:22.951844 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:22.951683    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" event={"ID":"fa99af78-e69d-4afb-b55c-95e3573282b8","Type":"ContainerDied","Data":"7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b"}
Jul 09 04:15:22.952153 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:22.952135    2582 scope.go:117] "RemoveContainer" containerID="7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b"
Jul 09 04:15:22.953204 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:22.953178    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:15:22.953266 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:22.953245    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:15:22.953302 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:22.953278    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:15:23.042398 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:23.042360    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:23.623580 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:23.623545    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:23.959033 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:23.958945    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" event={"ID":"fa99af78-e69d-4afb-b55c-95e3573282b8","Type":"ContainerStarted","Data":"048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460"}
Jul 09 04:15:23.971461 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:23.971435    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:24.972703 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:24.972669    2582 patch_prober.go:28] interesting pod/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 09 04:15:24.973109 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:24.972726    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 09 04:15:25.876717 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:25.876671    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:25.971014 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:25.970980    2582 patch_prober.go:28] interesting pod/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 09 04:15:25.971188 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:25.971051    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 09 04:15:26.951017 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:26.950990    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:26.971214 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:26.971177    2582 patch_prober.go:28] interesting pod/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 09 04:15:26.971392 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:26.971242    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 09 04:15:27.972029 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:27.971989    2582 patch_prober.go:28] interesting pod/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 09 04:15:27.972446 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:27.972080    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 09 04:15:28.975091 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:28.975061    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:28.976123 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:28.976100    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:31.643004 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:31.642957    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:32.065349 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.065307    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:32.166072 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.166038    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:32.267309 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.267275    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:32.369304 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.369212    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"]
Jul 09 04:15:32.369513 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.369488    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj" podUID="a28a97d9-edfd-40b1-bef6-afaa03691eb2" containerName="maas-api" containerID="cri-o://768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9" gracePeriod=30
Jul 09 04:15:32.391156 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.391124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:32.626293 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.626227    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:32.638077 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.638037    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a28a97d9-edfd-40b1-bef6-afaa03691eb2-maas-api-tls\") pod \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") "
Jul 09 04:15:32.638278 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.638121    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49tnt\" (UniqueName: \"kubernetes.io/projected/a28a97d9-edfd-40b1-bef6-afaa03691eb2-kube-api-access-49tnt\") pod \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") "
Jul 09 04:15:32.638358 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.638289    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28a97d9-edfd-40b1-bef6-afaa03691eb2-openshift-service-ca\") pod \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\" (UID: \"a28a97d9-edfd-40b1-bef6-afaa03691eb2\") "
Jul 09 04:15:32.638645 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.638618    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a28a97d9-edfd-40b1-bef6-afaa03691eb2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a28a97d9-edfd-40b1-bef6-afaa03691eb2" (UID: "a28a97d9-edfd-40b1-bef6-afaa03691eb2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:15:32.640480 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.640452    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28a97d9-edfd-40b1-bef6-afaa03691eb2-kube-api-access-49tnt" (OuterVolumeSpecName: "kube-api-access-49tnt") pod "a28a97d9-edfd-40b1-bef6-afaa03691eb2" (UID: "a28a97d9-edfd-40b1-bef6-afaa03691eb2"). InnerVolumeSpecName "kube-api-access-49tnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:15:32.640617 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.640554    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a28a97d9-edfd-40b1-bef6-afaa03691eb2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a28a97d9-edfd-40b1-bef6-afaa03691eb2" (UID: "a28a97d9-edfd-40b1-bef6-afaa03691eb2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:15:32.739163 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.739119    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a28a97d9-edfd-40b1-bef6-afaa03691eb2-maas-api-tls\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:32.739163 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.739154    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49tnt\" (UniqueName: \"kubernetes.io/projected/a28a97d9-edfd-40b1-bef6-afaa03691eb2-kube-api-access-49tnt\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:32.739163 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:32.739164    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28a97d9-edfd-40b1-bef6-afaa03691eb2-openshift-service-ca\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:33.005992 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:33.005899    2582 generic.go:356] "Generic (PLEG): container finished" podID="a28a97d9-edfd-40b1-bef6-afaa03691eb2" containerID="768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9" exitCode=0
Jul 09 04:15:33.005992 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:33.005946    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"
Jul 09 04:15:33.005992 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:33.005984    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj" event={"ID":"a28a97d9-edfd-40b1-bef6-afaa03691eb2","Type":"ContainerDied","Data":"768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9"}
Jul 09 04:15:33.006228 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:33.006036    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj" event={"ID":"a28a97d9-edfd-40b1-bef6-afaa03691eb2","Type":"ContainerDied","Data":"ccae14ab97f5665d924ffcf762fdf7664bd46daf73076d982503df66291cfffe"}
Jul 09 04:15:33.006228 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:33.006059    2582 scope.go:117] "RemoveContainer" containerID="768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9"
Jul 09 04:15:33.014828 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:33.014803    2582 scope.go:117] "RemoveContainer" containerID="768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9"
Jul 09 04:15:33.015142 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:15:33.015121    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9\": container with ID starting with 768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9 not found: ID does not exist" containerID="768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9"
Jul 09 04:15:33.015219 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:33.015156    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9"} err="failed to get container status \"768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9\": rpc error: code = NotFound desc = could not find container \"768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9\": container with ID starting with 768c0c6825818dc93046639f4441def0fbef83daa888fd73872db50c86a8f8f9 not found: ID does not exist"
Jul 09 04:15:33.030293 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:33.030254    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"]
Jul 09 04:15:33.033037 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:33.033011    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c44f7d-694dd48d87-6tvzj"]
Jul 09 04:15:33.714733 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:33.714687    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a28a97d9-edfd-40b1-bef6-afaa03691eb2" path="/var/lib/kubelet/pods/a28a97d9-edfd-40b1-bef6-afaa03691eb2/volumes"
Jul 09 04:15:38.084625 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:38.084587    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:38.290809 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:38.290768    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:38.315687 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:38.315640    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"]
Jul 09 04:15:38.315909 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:38.315881    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerName="istio-proxy" containerID="cri-o://048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460" gracePeriod=30
Jul 09 04:15:38.361376 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:38.361277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:38.750307 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:38.750210    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:39.064316 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.064270    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:39.179288 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.179254    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:39.280995 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.280956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:39.416061 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.415972    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"]
Jul 09 04:15:39.416226 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.416177    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5" podUID="81adc359-e062-49ca-aa92-4e49eb680056" containerName="maas-api" containerID="cri-o://36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f" gracePeriod=30
Jul 09 04:15:39.449051 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.449016    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:39.669395 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.669321    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:39.695117 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.695086    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81adc359-e062-49ca-aa92-4e49eb680056-openshift-service-ca\") pod \"81adc359-e062-49ca-aa92-4e49eb680056\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") "
Jul 09 04:15:39.695316 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.695155    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgnvl\" (UniqueName: \"kubernetes.io/projected/81adc359-e062-49ca-aa92-4e49eb680056-kube-api-access-hgnvl\") pod \"81adc359-e062-49ca-aa92-4e49eb680056\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") "
Jul 09 04:15:39.695316 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.695181    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81adc359-e062-49ca-aa92-4e49eb680056-maas-api-tls\") pod \"81adc359-e062-49ca-aa92-4e49eb680056\" (UID: \"81adc359-e062-49ca-aa92-4e49eb680056\") "
Jul 09 04:15:39.695519 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.695480    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81adc359-e062-49ca-aa92-4e49eb680056-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81adc359-e062-49ca-aa92-4e49eb680056" (UID: "81adc359-e062-49ca-aa92-4e49eb680056"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:15:39.697410 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.697376    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81adc359-e062-49ca-aa92-4e49eb680056-kube-api-access-hgnvl" (OuterVolumeSpecName: "kube-api-access-hgnvl") pod "81adc359-e062-49ca-aa92-4e49eb680056" (UID: "81adc359-e062-49ca-aa92-4e49eb680056"). InnerVolumeSpecName "kube-api-access-hgnvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:15:39.697410 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.697390    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81adc359-e062-49ca-aa92-4e49eb680056-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81adc359-e062-49ca-aa92-4e49eb680056" (UID: "81adc359-e062-49ca-aa92-4e49eb680056"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:15:39.796692 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.796655    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgnvl\" (UniqueName: \"kubernetes.io/projected/81adc359-e062-49ca-aa92-4e49eb680056-kube-api-access-hgnvl\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:39.796692 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.796692    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81adc359-e062-49ca-aa92-4e49eb680056-maas-api-tls\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:39.796949 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:39.796778    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81adc359-e062-49ca-aa92-4e49eb680056-openshift-service-ca\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:40.042522 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:40.042486    2582 generic.go:356] "Generic (PLEG): container finished" podID="81adc359-e062-49ca-aa92-4e49eb680056" containerID="36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f" exitCode=0
Jul 09 04:15:40.042720 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:40.042541    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"
Jul 09 04:15:40.042720 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:40.042559    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5" event={"ID":"81adc359-e062-49ca-aa92-4e49eb680056","Type":"ContainerDied","Data":"36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f"}
Jul 09 04:15:40.042720 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:40.042603    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5" event={"ID":"81adc359-e062-49ca-aa92-4e49eb680056","Type":"ContainerDied","Data":"a92aa77d3604db47520e5d5279ffc9fd10abfa7d2dbb37ecb536467efffe9d9d"}
Jul 09 04:15:40.042720 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:40.042625    2582 scope.go:117] "RemoveContainer" containerID="36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f"
Jul 09 04:15:40.051332 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:40.051309    2582 scope.go:117] "RemoveContainer" containerID="36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f"
Jul 09 04:15:40.051683 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:15:40.051664    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f\": container with ID starting with 36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f not found: ID does not exist" containerID="36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f"
Jul 09 04:15:40.051751 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:40.051695    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f"} err="failed to get container status \"36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f\": rpc error: code = NotFound desc = could not find container \"36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f\": container with ID starting with 36fd425bfec0735578201a3a5ba893d5fa35fc2dbda65ae07adcae17dc28c53f not found: ID does not exist"
Jul 09 04:15:40.063430 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:40.063394    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"]
Jul 09 04:15:40.067323 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:40.067295    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f007b3-5859884c44-958h5"]
Jul 09 04:15:41.714831 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:41.714799    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81adc359-e062-49ca-aa92-4e49eb680056" path="/var/lib/kubelet/pods/81adc359-e062-49ca-aa92-4e49eb680056/volumes"
Jul 09 04:15:43.574380 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.574353    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:43.629447 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.629354    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-podinfo\") pod \"fa99af78-e69d-4afb-b55c-95e3573282b8\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") "
Jul 09 04:15:43.629447 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.629419    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-token\") pod \"fa99af78-e69d-4afb-b55c-95e3573282b8\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") "
Jul 09 04:15:43.629679 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.629479    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-credential-socket\") pod \"fa99af78-e69d-4afb-b55c-95e3573282b8\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") "
Jul 09 04:15:43.629679 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.629510    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-envoy\") pod \"fa99af78-e69d-4afb-b55c-95e3573282b8\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") "
Jul 09 04:15:43.629679 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.629540    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfncm\" (UniqueName: \"kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-kube-api-access-rfncm\") pod \"fa99af78-e69d-4afb-b55c-95e3573282b8\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") "
Jul 09 04:15:43.629679 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.629591    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-certs\") pod \"fa99af78-e69d-4afb-b55c-95e3573282b8\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") "
Jul 09 04:15:43.629679 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.629618    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-socket\") pod \"fa99af78-e69d-4afb-b55c-95e3573282b8\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") "
Jul 09 04:15:43.629679 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.629670    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-data\") pod \"fa99af78-e69d-4afb-b55c-95e3573282b8\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") "
Jul 09 04:15:43.630042 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.629696    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa99af78-e69d-4afb-b55c-95e3573282b8-istiod-ca-cert\") pod \"fa99af78-e69d-4afb-b55c-95e3573282b8\" (UID: \"fa99af78-e69d-4afb-b55c-95e3573282b8\") "
Jul 09 04:15:43.630042 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.629943    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fa99af78-e69d-4afb-b55c-95e3573282b8" (UID: "fa99af78-e69d-4afb-b55c-95e3573282b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:15:43.630154 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.630077    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-credential-socket\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:43.630253 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.630103    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fa99af78-e69d-4afb-b55c-95e3573282b8" (UID: "fa99af78-e69d-4afb-b55c-95e3573282b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:15:43.630319 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.630177    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fa99af78-e69d-4afb-b55c-95e3573282b8" (UID: "fa99af78-e69d-4afb-b55c-95e3573282b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:15:43.630319 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.630292    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa99af78-e69d-4afb-b55c-95e3573282b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fa99af78-e69d-4afb-b55c-95e3573282b8" (UID: "fa99af78-e69d-4afb-b55c-95e3573282b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:15:43.630689 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.630668    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "fa99af78-e69d-4afb-b55c-95e3573282b8" (UID: "fa99af78-e69d-4afb-b55c-95e3573282b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:15:43.631900 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.631853    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa99af78-e69d-4afb-b55c-95e3573282b8" (UID: "fa99af78-e69d-4afb-b55c-95e3573282b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:15:43.632030 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.631992    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fa99af78-e69d-4afb-b55c-95e3573282b8" (UID: "fa99af78-e69d-4afb-b55c-95e3573282b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 04:15:43.632193 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.632161    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fa99af78-e69d-4afb-b55c-95e3573282b8" (UID: "fa99af78-e69d-4afb-b55c-95e3573282b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:15:43.632244 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.632216    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-kube-api-access-rfncm" (OuterVolumeSpecName: "kube-api-access-rfncm") pod "fa99af78-e69d-4afb-b55c-95e3573282b8" (UID: "fa99af78-e69d-4afb-b55c-95e3573282b8"). InnerVolumeSpecName "kube-api-access-rfncm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:15:43.730447 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.730418    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfncm\" (UniqueName: \"kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-kube-api-access-rfncm\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:43.730447 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.730443    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-certs\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:43.730447 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.730453    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-workload-socket\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:43.730671 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.730461    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-data\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:43.730671 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.730470    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa99af78-e69d-4afb-b55c-95e3573282b8-istiod-ca-cert\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:43.730671 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.730477    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-podinfo\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:43.730671 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.730484    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-token\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:43.730671 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:43.730493    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa99af78-e69d-4afb-b55c-95e3573282b8-istio-envoy\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:15:44.064532 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.064497    2582 generic.go:356] "Generic (PLEG): container finished" podID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerID="048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460" exitCode=0
Jul 09 04:15:44.064708 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.064557    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"
Jul 09 04:15:44.064708 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.064565    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" event={"ID":"fa99af78-e69d-4afb-b55c-95e3573282b8","Type":"ContainerDied","Data":"048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460"}
Jul 09 04:15:44.064708 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.064609    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2" event={"ID":"fa99af78-e69d-4afb-b55c-95e3573282b8","Type":"ContainerDied","Data":"8467fad1d2ac8ca50ebe22bc640953b07cc9237421757a7e2fb89ffe4f02d876"}
Jul 09 04:15:44.064708 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.064630    2582 scope.go:117] "RemoveContainer" containerID="048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460"
Jul 09 04:15:44.073428 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.073398    2582 scope.go:117] "RemoveContainer" containerID="7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b"
Jul 09 04:15:44.081694 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.081664    2582 scope.go:117] "RemoveContainer" containerID="048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460"
Jul 09 04:15:44.082103 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:15:44.082076    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460\": container with ID starting with 048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460 not found: ID does not exist" containerID="048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460"
Jul 09 04:15:44.082201 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.082114    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460"} err="failed to get container status \"048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460\": rpc error: code = NotFound desc = could not find container \"048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460\": container with ID starting with 048b1bc4e1b9e102f93523787a894a001a87816649d37a17e41cf8abcb2c1460 not found: ID does not exist"
Jul 09 04:15:44.082201 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.082134    2582 scope.go:117] "RemoveContainer" containerID="7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b"
Jul 09 04:15:44.082447 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:15:44.082416    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b\": container with ID starting with 7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b not found: ID does not exist" containerID="7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b"
Jul 09 04:15:44.082619 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.082453    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b"} err="failed to get container status \"7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b\": rpc error: code = NotFound desc = could not find container \"7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b\": container with ID starting with 7e5e1b6a3c77801d067a6deda0f9b0ebd1fecb87be92cebcb3db3e2b783a386b not found: ID does not exist"
Jul 09 04:15:44.088673 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.088643    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"]
Jul 09 04:15:44.094661 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.094630    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c44f7d-openshift-default-7ccd4864d7-z7gz2"]
Jul 09 04:15:44.936850 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:44.936814    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:45.037628 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:45.037581    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:45.209716 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:45.209623    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:45.713109 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:45.713077    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" path="/var/lib/kubelet/pods/fa99af78-e69d-4afb-b55c-95e3573282b8/volumes"
Jul 09 04:15:45.733731 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:45.733698    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:46.004793 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.004708    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"]
Jul 09 04:15:46.005239 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.005091    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerName="istio-proxy"
Jul 09 04:15:46.005239 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.005110    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerName="istio-proxy"
Jul 09 04:15:46.005239 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.005131    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerName="istio-proxy"
Jul 09 04:15:46.005239 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.005139    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerName="istio-proxy"
Jul 09 04:15:46.005239 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.005177    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a28a97d9-edfd-40b1-bef6-afaa03691eb2" containerName="maas-api"
Jul 09 04:15:46.005239 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.005186    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28a97d9-edfd-40b1-bef6-afaa03691eb2" containerName="maas-api"
Jul 09 04:15:46.005239 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.005196    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81adc359-e062-49ca-aa92-4e49eb680056" containerName="maas-api"
Jul 09 04:15:46.005239 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.005204    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="81adc359-e062-49ca-aa92-4e49eb680056" containerName="maas-api"
Jul 09 04:15:46.005536 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.005279    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerName="istio-proxy"
Jul 09 04:15:46.005536 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.005297    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a28a97d9-edfd-40b1-bef6-afaa03691eb2" containerName="maas-api"
Jul 09 04:15:46.005536 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.005305    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="81adc359-e062-49ca-aa92-4e49eb680056" containerName="maas-api"
Jul 09 04:15:46.008195 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.008176    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.011119 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.011091    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a3bdc1-openshift-default-dockercfg-9njm9\""
Jul 09 04:15:46.019677 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.019643    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"]
Jul 09 04:15:46.033208 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.033162    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:46.050340 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.050302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-credential-socket\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.050528 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.050355    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-socket\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.050528 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.050384    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-certs\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.050528 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.050443    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssmct\" (UniqueName: \"kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-kube-api-access-ssmct\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.050528 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.050485    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-data\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.050528 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.050515    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-token\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.050777 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.050559    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-podinfo\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.050777 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.050585    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-envoy\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.050777 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.050654    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b36bc16-de79-4aae-aaa6-565d03a081de-istiod-ca-cert\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151174 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151126    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b36bc16-de79-4aae-aaa6-565d03a081de-istiod-ca-cert\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151392 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-credential-socket\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151392 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151230    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-socket\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151392 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-certs\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151392 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151302    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssmct\" (UniqueName: \"kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-kube-api-access-ssmct\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151392 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151346    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-data\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151392 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-token\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151706 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151657    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-socket\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151706 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151685    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-credential-socket\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151907 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151714    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-podinfo\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151907 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-envoy\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151907 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151761    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-certs\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.151907 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151812    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-data\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.152059 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.151999    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b36bc16-de79-4aae-aaa6-565d03a081de-istiod-ca-cert\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.153986 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.153956    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-envoy\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.154126 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.154060    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-podinfo\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.159960 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.159901    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-token\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.160072 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.160012    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssmct\" (UniqueName: \"kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-kube-api-access-ssmct\") pod \"e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") " pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.320680 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.320642    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:46.394402 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.394367    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:46.463840 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.463814    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"]
Jul 09 04:15:46.465479 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:15:46.465442    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b36bc16_de79_4aae_aaa6_565d03a081de.slice/crio-3af5ea54ae06466ca7bade075598439efc8bf9a3756f4f532958334a4b2ffdbe WatchSource:0}: Error finding container 3af5ea54ae06466ca7bade075598439efc8bf9a3756f4f532958334a4b2ffdbe: Status 404 returned error can't find the container with id 3af5ea54ae06466ca7bade075598439efc8bf9a3756f4f532958334a4b2ffdbe
Jul 09 04:15:46.467607 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.467571    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:15:46.467698 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.467640    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:15:46.467698 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:46.467674    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:15:47.082954 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:47.082908    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" event={"ID":"9b36bc16-de79-4aae-aaa6-565d03a081de","Type":"ContainerStarted","Data":"eb87f2914eab378289b11fa4f72cc074726120485eb2e9b0e0f1a1850e72edce"}
Jul 09 04:15:47.083474 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:47.082976    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" event={"ID":"9b36bc16-de79-4aae-aaa6-565d03a081de","Type":"ContainerStarted","Data":"3af5ea54ae06466ca7bade075598439efc8bf9a3756f4f532958334a4b2ffdbe"}
Jul 09 04:15:47.105361 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:47.105306    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" podStartSLOduration=2.105290142 podStartE2EDuration="2.105290142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:47.10331257 +0000 UTC m=+1963.922568348" watchObservedRunningTime="2026-07-09 04:15:47.105290142 +0000 UTC m=+1963.924545919"
Jul 09 04:15:47.321818 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:47.321780    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:47.327290 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:47.327258    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:48.088343 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:48.088308    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:48.089504 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:48.089482    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:15:49.034088 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.034054    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:49.102990 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.102957    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"]
Jul 09 04:15:49.103477 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.103317    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa99af78-e69d-4afb-b55c-95e3573282b8" containerName="istio-proxy"
Jul 09 04:15:49.105473 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.105451    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:49.108504 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.108480    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-a3bdc1-serving-cert\""
Jul 09 04:15:49.119259 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.119231    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"]
Jul 09 04:15:49.177602 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.177557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45h2n\" (UniqueName: \"kubernetes.io/projected/da8ef171-235a-4695-8e40-e00dad8226e7-kube-api-access-45h2n\") pod \"maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:49.177602 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.177604    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8ef171-235a-4695-8e40-e00dad8226e7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:49.177913 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.177704    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da8ef171-235a-4695-8e40-e00dad8226e7-maas-api-tls\") pod \"maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:49.189542 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.189501    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:49.279046 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.279003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45h2n\" (UniqueName: \"kubernetes.io/projected/da8ef171-235a-4695-8e40-e00dad8226e7-kube-api-access-45h2n\") pod \"maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:49.279046 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.279049    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8ef171-235a-4695-8e40-e00dad8226e7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:49.279278 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.279093    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da8ef171-235a-4695-8e40-e00dad8226e7-maas-api-tls\") pod \"maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:49.279795 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.279767    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8ef171-235a-4695-8e40-e00dad8226e7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:49.281771 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.281745    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da8ef171-235a-4695-8e40-e00dad8226e7-maas-api-tls\") pod \"maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:49.289145 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.289071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:49.289510 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.289493    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45h2n\" (UniqueName: \"kubernetes.io/projected/da8ef171-235a-4695-8e40-e00dad8226e7-kube-api-access-45h2n\") pod \"maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:49.417001 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.416963    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:49.563218 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.563190    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"]
Jul 09 04:15:49.564429 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:15:49.564398    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda8ef171_235a_4695_8e40_e00dad8226e7.slice/crio-9f3d1fbc82b296c3dbce21fac2075261660a045cb951c9c2c49ac46d8dd644aa WatchSource:0}: Error finding container 9f3d1fbc82b296c3dbce21fac2075261660a045cb951c9c2c49ac46d8dd644aa: Status 404 returned error can't find the container with id 9f3d1fbc82b296c3dbce21fac2075261660a045cb951c9c2c49ac46d8dd644aa
Jul 09 04:15:49.597200 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.597163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:49.711588 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:49.711548    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:50.103934 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:50.103899    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw" event={"ID":"da8ef171-235a-4695-8e40-e00dad8226e7","Type":"ContainerStarted","Data":"13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11"}
Jul 09 04:15:50.104425 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:50.103963    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw" event={"ID":"da8ef171-235a-4695-8e40-e00dad8226e7","Type":"ContainerStarted","Data":"9f3d1fbc82b296c3dbce21fac2075261660a045cb951c9c2c49ac46d8dd644aa"}
Jul 09 04:15:50.104425 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:50.103994    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:53.479200 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:53.479107    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:53.801424 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:53.801382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:54.097067 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:54.096974    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:54.382904 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:54.382772    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:56.115666 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:56.115637    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:15:56.133731 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:56.133680    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw" podStartSLOduration=6.7956217169999995 podStartE2EDuration="7.133661077s" totalImagesPullingTime="338.03936ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:50.126482225 +0000 UTC m=+1966.945738001" watchObservedRunningTime="2026-07-09 04:15:56.133661077 +0000 UTC m=+1972.952916854"
Jul 09 04:15:57.369615 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.369578    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:57.444562 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.444524    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"]
Jul 09 04:15:57.448539 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.448511    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:57.451626 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.451600    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-90e242-serving-cert\""
Jul 09 04:15:57.466639 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.466604    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"]
Jul 09 04:15:57.519463 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.519416    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:57.556520 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.556480    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27906d5c-57dd-4aa6-98dc-13b017ebf669-maas-api-tls\") pod \"maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:57.556747 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.556540    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27906d5c-57dd-4aa6-98dc-13b017ebf669-openshift-service-ca\") pod \"maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:57.556747 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.556562    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kplg9\" (UniqueName: \"kubernetes.io/projected/27906d5c-57dd-4aa6-98dc-13b017ebf669-kube-api-access-kplg9\") pod \"maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:57.629240 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.629133    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:57.657162 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.657107    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27906d5c-57dd-4aa6-98dc-13b017ebf669-maas-api-tls\") pod \"maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:57.657374 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.657221    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27906d5c-57dd-4aa6-98dc-13b017ebf669-openshift-service-ca\") pod \"maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:57.657374 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.657253    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kplg9\" (UniqueName: \"kubernetes.io/projected/27906d5c-57dd-4aa6-98dc-13b017ebf669-kube-api-access-kplg9\") pod \"maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:57.657374 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:15:57.657259    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-serving-cert: secret "maas-api-e2e-shared-b-90e242-serving-cert" not found
Jul 09 04:15:57.657374 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:15:57.657323    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27906d5c-57dd-4aa6-98dc-13b017ebf669-maas-api-tls podName:27906d5c-57dd-4aa6-98dc-13b017ebf669 nodeName:}" failed. No retries permitted until 2026-07-09 04:15:58.157304886 +0000 UTC m=+1974.976560642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/27906d5c-57dd-4aa6-98dc-13b017ebf669-maas-api-tls") pod "maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv" (UID: "27906d5c-57dd-4aa6-98dc-13b017ebf669") : secret "maas-api-e2e-shared-b-90e242-serving-cert" not found
Jul 09 04:15:57.657960 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.657943    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27906d5c-57dd-4aa6-98dc-13b017ebf669-openshift-service-ca\") pod \"maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:57.677110 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.677072    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kplg9\" (UniqueName: \"kubernetes.io/projected/27906d5c-57dd-4aa6-98dc-13b017ebf669-kube-api-access-kplg9\") pod \"maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:57.863178 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:57.863127    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:58.161281 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:58.161242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27906d5c-57dd-4aa6-98dc-13b017ebf669-maas-api-tls\") pod \"maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:58.163831 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:58.163799    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27906d5c-57dd-4aa6-98dc-13b017ebf669-maas-api-tls\") pod \"maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:58.238309 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:58.238268    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:58.340587 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:58.340549    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:15:58.359995 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:58.359959    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:58.559473 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:58.559445    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"]
Jul 09 04:15:58.562556 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:15:58.562527    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27906d5c_57dd_4aa6_98dc_13b017ebf669.slice/crio-58f089156409166890ecdf9ee976d233973a79ca843578d2876671998ff3fa5d WatchSource:0}: Error finding container 58f089156409166890ecdf9ee976d233973a79ca843578d2876671998ff3fa5d: Status 404 returned error can't find the container with id 58f089156409166890ecdf9ee976d233973a79ca843578d2876671998ff3fa5d
Jul 09 04:15:59.153802 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:59.153765    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv" event={"ID":"27906d5c-57dd-4aa6-98dc-13b017ebf669","Type":"ContainerStarted","Data":"1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065"}
Jul 09 04:15:59.154014 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:59.153820    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv" event={"ID":"27906d5c-57dd-4aa6-98dc-13b017ebf669","Type":"ContainerStarted","Data":"58f089156409166890ecdf9ee976d233973a79ca843578d2876671998ff3fa5d"}
Jul 09 04:15:59.154014 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:59.153904    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:15:59.173023 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:15:59.172911    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv" podStartSLOduration=1.8159322709999999 podStartE2EDuration="2.172893826s" totalImagesPullingTime="356.961555ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:59.171442218 +0000 UTC m=+1975.990697997" watchObservedRunningTime="2026-07-09 04:15:59.172893826 +0000 UTC m=+1975.992149618"
Jul 09 04:16:00.020479 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:00.020443    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"]
Jul 09 04:16:00.020930 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:00.020648    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx" podUID="042df7af-eaf9-435b-9c72-75474ab8029d" containerName="cleanup" containerID="cri-o://95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4" gracePeriod=30
Jul 09 04:16:01.777413 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:01.777376    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:16:04.130112 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:04.130019    2582 patch_prober.go:28] interesting pod/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 09 04:16:04.130499 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:04.130102    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 09 04:16:05.165099 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:05.165066    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:16:05.189341 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:05.189299    2582 generic.go:356] "Generic (PLEG): container finished" podID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerID="eb87f2914eab378289b11fa4f72cc074726120485eb2e9b0e0f1a1850e72edce" exitCode=0
Jul 09 04:16:05.189549 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:05.189366    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" event={"ID":"9b36bc16-de79-4aae-aaa6-565d03a081de","Type":"ContainerDied","Data":"eb87f2914eab378289b11fa4f72cc074726120485eb2e9b0e0f1a1850e72edce"}
Jul 09 04:16:05.189960 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:05.189940    2582 scope.go:117] "RemoveContainer" containerID="eb87f2914eab378289b11fa4f72cc074726120485eb2e9b0e0f1a1850e72edce"
Jul 09 04:16:05.191394 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:05.191362    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:16:05.191463 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:05.191451    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:16:05.191520 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:05.191504    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 09 04:16:06.200459 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:06.200416    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" event={"ID":"9b36bc16-de79-4aae-aaa6-565d03a081de","Type":"ContainerStarted","Data":"84f4750ea78f41ef20cf624dd05054aed8736707d85b5d6fea850a754123f3f6"}
Jul 09 04:16:06.321715 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:06.321679    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:16:06.321996 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:06.321972    2582 patch_prober.go:28] interesting pod/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 09 04:16:06.322055 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:06.322029    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 09 04:16:07.321658 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:07.321617    2582 patch_prober.go:28] interesting pod/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 09 04:16:07.322081 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:07.321674    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 09 04:16:08.321755 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:08.321707    2582 patch_prober.go:28] interesting pod/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 09 04:16:08.322206 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:08.321779    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 09 04:16:09.321969 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:09.321932    2582 patch_prober.go:28] interesting pod/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 09 04:16:09.322396 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:09.322006    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 09 04:16:10.325363 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:10.325329    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:16:10.325789 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:10.325669    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:16:10.326307 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:10.326292    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:16:10.504546 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:10.504508    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:16:10.619955 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:10.619850    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:16:19.931812 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:19.931779    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:16:20.033122 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:20.033079    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:16:30.212033 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.212007    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"
Jul 09 04:16:30.249050 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.249012    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hd8sv\" (UniqueName: \"kubernetes.io/projected/042df7af-eaf9-435b-9c72-75474ab8029d-kube-api-access-hd8sv\") pod \"042df7af-eaf9-435b-9c72-75474ab8029d\" (UID: \"042df7af-eaf9-435b-9c72-75474ab8029d\") "
Jul 09 04:16:30.251203 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.251169    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/042df7af-eaf9-435b-9c72-75474ab8029d-kube-api-access-hd8sv" (OuterVolumeSpecName: "kube-api-access-hd8sv") pod "042df7af-eaf9-435b-9c72-75474ab8029d" (UID: "042df7af-eaf9-435b-9c72-75474ab8029d"). InnerVolumeSpecName "kube-api-access-hd8sv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:16:30.323181 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.323150    2582 generic.go:356] "Generic (PLEG): container finished" podID="042df7af-eaf9-435b-9c72-75474ab8029d" containerID="95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4" exitCode=137
Jul 09 04:16:30.323368 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.323200    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx" event={"ID":"042df7af-eaf9-435b-9c72-75474ab8029d","Type":"ContainerDied","Data":"95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4"}
Jul 09 04:16:30.323368 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.323205    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"
Jul 09 04:16:30.323368 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.323241    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx" event={"ID":"042df7af-eaf9-435b-9c72-75474ab8029d","Type":"ContainerDied","Data":"2f409f6bdb7bfa8962fd89a7d965ee265d1eb7cbd5030b78a7bf4d2138ef0c5e"}
Jul 09 04:16:30.323368 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.323263    2582 scope.go:117] "RemoveContainer" containerID="95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4"
Jul 09 04:16:30.332371 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.332353    2582 scope.go:117] "RemoveContainer" containerID="95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4"
Jul 09 04:16:30.332654 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:16:30.332636    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4\": container with ID starting with 95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4 not found: ID does not exist" containerID="95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4"
Jul 09 04:16:30.332746 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.332662    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4"} err="failed to get container status \"95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4\": rpc error: code = NotFound desc = could not find container \"95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4\": container with ID starting with 95c2956a2e5040444a56aaa249cb3fee16065cc0a0a5674ac8f518d2e355c1f4 not found: ID does not exist"
Jul 09 04:16:30.349887 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.349837    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd8sv\" (UniqueName: \"kubernetes.io/projected/042df7af-eaf9-435b-9c72-75474ab8029d-kube-api-access-hd8sv\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:16:30.354961 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.354930    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"]
Jul 09 04:16:30.360539 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:30.360512    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726175-jvqtx"]
Jul 09 04:16:31.714012 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:16:31.713968    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="042df7af-eaf9-435b-9c72-75474ab8029d" path="/var/lib/kubelet/pods/042df7af-eaf9-435b-9c72-75474ab8029d/volumes"
Jul 09 04:17:21.040540 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:17:21.040506    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:17:47.726929 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:17:47.726890    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:17:47.826683 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:17:47.826647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:18:03.746191 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:18:03.746153    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 04:18:03.748730 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:18:03.748709    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 04:18:03.750158 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:18:03.750141    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2pkz_98de9f69-0044-4adc-879b-5eb059d178ad/ovn-acl-logging/0.log"
Jul 09 04:18:03.752490 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:18:03.752465    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-134-164.ec2.internal_3353beadccb3550d362986f67bd77010/kube-rbac-proxy-crio/1.log"
Jul 09 04:19:06.648198 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:06.648106    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:07.113546 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.113511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:07.214566 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.214517    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:07.314279 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.314243    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:07.424098 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.424004    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"]
Jul 09 04:19:07.424304 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.424266    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw" podUID="da8ef171-235a-4695-8e40-e00dad8226e7" containerName="maas-api" containerID="cri-o://13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11" gracePeriod=30
Jul 09 04:19:07.452916 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.452850    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:07.671029 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.671000    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:19:07.802504 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.802469    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45h2n\" (UniqueName: \"kubernetes.io/projected/da8ef171-235a-4695-8e40-e00dad8226e7-kube-api-access-45h2n\") pod \"da8ef171-235a-4695-8e40-e00dad8226e7\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") "
Jul 09 04:19:07.802504 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.802504    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8ef171-235a-4695-8e40-e00dad8226e7-openshift-service-ca\") pod \"da8ef171-235a-4695-8e40-e00dad8226e7\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") "
Jul 09 04:19:07.802768 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.802530    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da8ef171-235a-4695-8e40-e00dad8226e7-maas-api-tls\") pod \"da8ef171-235a-4695-8e40-e00dad8226e7\" (UID: \"da8ef171-235a-4695-8e40-e00dad8226e7\") "
Jul 09 04:19:07.802910 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.802855    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da8ef171-235a-4695-8e40-e00dad8226e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da8ef171-235a-4695-8e40-e00dad8226e7" (UID: "da8ef171-235a-4695-8e40-e00dad8226e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:19:07.804780 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.804751    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da8ef171-235a-4695-8e40-e00dad8226e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da8ef171-235a-4695-8e40-e00dad8226e7" (UID: "da8ef171-235a-4695-8e40-e00dad8226e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:19:07.804888 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.804751    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da8ef171-235a-4695-8e40-e00dad8226e7-kube-api-access-45h2n" (OuterVolumeSpecName: "kube-api-access-45h2n") pod "da8ef171-235a-4695-8e40-e00dad8226e7" (UID: "da8ef171-235a-4695-8e40-e00dad8226e7"). InnerVolumeSpecName "kube-api-access-45h2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:19:07.903670 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.903622    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45h2n\" (UniqueName: \"kubernetes.io/projected/da8ef171-235a-4695-8e40-e00dad8226e7-kube-api-access-45h2n\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:07.903670 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.903660    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8ef171-235a-4695-8e40-e00dad8226e7-openshift-service-ca\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:07.903899 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:07.903695    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da8ef171-235a-4695-8e40-e00dad8226e7-maas-api-tls\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:08.110016 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:08.109931    2582 generic.go:356] "Generic (PLEG): container finished" podID="da8ef171-235a-4695-8e40-e00dad8226e7" containerID="13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11" exitCode=0
Jul 09 04:19:08.110016 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:08.109982    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"
Jul 09 04:19:08.110211 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:08.110017    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw" event={"ID":"da8ef171-235a-4695-8e40-e00dad8226e7","Type":"ContainerDied","Data":"13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11"}
Jul 09 04:19:08.110211 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:08.110079    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw" event={"ID":"da8ef171-235a-4695-8e40-e00dad8226e7","Type":"ContainerDied","Data":"9f3d1fbc82b296c3dbce21fac2075261660a045cb951c9c2c49ac46d8dd644aa"}
Jul 09 04:19:08.110211 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:08.110106    2582 scope.go:117] "RemoveContainer" containerID="13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11"
Jul 09 04:19:08.119084 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:08.119064    2582 scope.go:117] "RemoveContainer" containerID="13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11"
Jul 09 04:19:08.119382 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:19:08.119364    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11\": container with ID starting with 13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11 not found: ID does not exist" containerID="13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11"
Jul 09 04:19:08.119445 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:08.119396    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11"} err="failed to get container status \"13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11\": rpc error: code = NotFound desc = could not find container \"13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11\": container with ID starting with 13521c7f36cb62a5aa5ede353b9676254919dd71731bd625c036d4fc50cdca11 not found: ID does not exist"
Jul 09 04:19:08.132837 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:08.132797    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"]
Jul 09 04:19:08.134756 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:08.134732    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a3bdc1-5fc877b746-bjnqw"]
Jul 09 04:19:09.714190 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:09.714159    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da8ef171-235a-4695-8e40-e00dad8226e7" path="/var/lib/kubelet/pods/da8ef171-235a-4695-8e40-e00dad8226e7/volumes"
Jul 09 04:19:12.687382 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:12.687345    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:12.751272 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:12.751239    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:12.965880 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:12.965777    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"]
Jul 09 04:19:12.966209 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:12.966179    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy" containerID="cri-o://84f4750ea78f41ef20cf624dd05054aed8736707d85b5d6fea850a754123f3f6" gracePeriod=30
Jul 09 04:19:12.990568 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:12.990530    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:13.406540 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:13.406495    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:13.836102 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:13.836066    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:13.936833 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:13.936795    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:14.035969 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.035923    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:14.149472 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.149387    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"]
Jul 09 04:19:14.149649 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.149624    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv" podUID="27906d5c-57dd-4aa6-98dc-13b017ebf669" containerName="maas-api" containerID="cri-o://1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065" gracePeriod=30
Jul 09 04:19:14.182025 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.181990    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:14.392722 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.392696    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:19:14.457638 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.457534    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27906d5c-57dd-4aa6-98dc-13b017ebf669-maas-api-tls\") pod \"27906d5c-57dd-4aa6-98dc-13b017ebf669\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") "
Jul 09 04:19:14.457638 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.457576    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27906d5c-57dd-4aa6-98dc-13b017ebf669-openshift-service-ca\") pod \"27906d5c-57dd-4aa6-98dc-13b017ebf669\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") "
Jul 09 04:19:14.457638 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.457617    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kplg9\" (UniqueName: \"kubernetes.io/projected/27906d5c-57dd-4aa6-98dc-13b017ebf669-kube-api-access-kplg9\") pod \"27906d5c-57dd-4aa6-98dc-13b017ebf669\" (UID: \"27906d5c-57dd-4aa6-98dc-13b017ebf669\") "
Jul 09 04:19:14.458036 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.458010    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27906d5c-57dd-4aa6-98dc-13b017ebf669-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "27906d5c-57dd-4aa6-98dc-13b017ebf669" (UID: "27906d5c-57dd-4aa6-98dc-13b017ebf669"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:19:14.459794 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.459771    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27906d5c-57dd-4aa6-98dc-13b017ebf669-kube-api-access-kplg9" (OuterVolumeSpecName: "kube-api-access-kplg9") pod "27906d5c-57dd-4aa6-98dc-13b017ebf669" (UID: "27906d5c-57dd-4aa6-98dc-13b017ebf669"). InnerVolumeSpecName "kube-api-access-kplg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:19:14.459888 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.459777    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27906d5c-57dd-4aa6-98dc-13b017ebf669-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "27906d5c-57dd-4aa6-98dc-13b017ebf669" (UID: "27906d5c-57dd-4aa6-98dc-13b017ebf669"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 04:19:14.558703 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.558662    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27906d5c-57dd-4aa6-98dc-13b017ebf669-maas-api-tls\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:14.558703 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.558697    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27906d5c-57dd-4aa6-98dc-13b017ebf669-openshift-service-ca\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:14.558703 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:14.558708    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kplg9\" (UniqueName: \"kubernetes.io/projected/27906d5c-57dd-4aa6-98dc-13b017ebf669-kube-api-access-kplg9\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:15.146204 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:15.146171    2582 generic.go:356] "Generic (PLEG): container finished" podID="27906d5c-57dd-4aa6-98dc-13b017ebf669" containerID="1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065" exitCode=0
Jul 09 04:19:15.146772 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:15.146228    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"
Jul 09 04:19:15.146772 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:15.146238    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv" event={"ID":"27906d5c-57dd-4aa6-98dc-13b017ebf669","Type":"ContainerDied","Data":"1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065"}
Jul 09 04:19:15.146772 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:15.146286    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv" event={"ID":"27906d5c-57dd-4aa6-98dc-13b017ebf669","Type":"ContainerDied","Data":"58f089156409166890ecdf9ee976d233973a79ca843578d2876671998ff3fa5d"}
Jul 09 04:19:15.146772 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:15.146309    2582 scope.go:117] "RemoveContainer" containerID="1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065"
Jul 09 04:19:15.155379 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:15.155208    2582 scope.go:117] "RemoveContainer" containerID="1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065"
Jul 09 04:19:15.155500 ip-10-0-134-164 kubenswrapper[2582]: E0709 04:19:15.155478    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065\": container with ID starting with 1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065 not found: ID does not exist" containerID="1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065"
Jul 09 04:19:15.155561 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:15.155513    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065"} err="failed to get container status \"1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065\": rpc error: code = NotFound desc = could not find container \"1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065\": container with ID starting with 1124c16fb28f705fcaf292686c1ae7a455d0eb7fb8660ea4d52c32b194b06065 not found: ID does not exist"
Jul 09 04:19:15.167694 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:15.167661    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"]
Jul 09 04:19:15.173255 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:15.173228    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-90e242-87f6c7db5-7zqhv"]
Jul 09 04:19:15.714148 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:15.714116    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27906d5c-57dd-4aa6-98dc-13b017ebf669" path="/var/lib/kubelet/pods/27906d5c-57dd-4aa6-98dc-13b017ebf669/volumes"
Jul 09 04:19:18.164553 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.164517    2582 generic.go:356] "Generic (PLEG): container finished" podID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerID="84f4750ea78f41ef20cf624dd05054aed8736707d85b5d6fea850a754123f3f6" exitCode=0
Jul 09 04:19:18.164553 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.164542    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" event={"ID":"9b36bc16-de79-4aae-aaa6-565d03a081de","Type":"ContainerDied","Data":"84f4750ea78f41ef20cf624dd05054aed8736707d85b5d6fea850a754123f3f6"}
Jul 09 04:19:18.165039 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.164615    2582 scope.go:117] "RemoveContainer" containerID="eb87f2914eab378289b11fa4f72cc074726120485eb2e9b0e0f1a1850e72edce"
Jul 09 04:19:18.212449 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.212411    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:19:18.290924 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.290893    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-envoy\") pod \"9b36bc16-de79-4aae-aaa6-565d03a081de\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") "
Jul 09 04:19:18.291102 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.290939    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-socket\") pod \"9b36bc16-de79-4aae-aaa6-565d03a081de\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") "
Jul 09 04:19:18.291102 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.290977    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssmct\" (UniqueName: \"kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-kube-api-access-ssmct\") pod \"9b36bc16-de79-4aae-aaa6-565d03a081de\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") "
Jul 09 04:19:18.291102 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.291033    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-credential-socket\") pod \"9b36bc16-de79-4aae-aaa6-565d03a081de\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") "
Jul 09 04:19:18.291102 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.291087    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-data\") pod \"9b36bc16-de79-4aae-aaa6-565d03a081de\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") "
Jul 09 04:19:18.291297 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.291115    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-certs\") pod \"9b36bc16-de79-4aae-aaa6-565d03a081de\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") "
Jul 09 04:19:18.291297 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.291141    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b36bc16-de79-4aae-aaa6-565d03a081de-istiod-ca-cert\") pod \"9b36bc16-de79-4aae-aaa6-565d03a081de\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") "
Jul 09 04:19:18.291405 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.291287    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b36bc16-de79-4aae-aaa6-565d03a081de" (UID: "9b36bc16-de79-4aae-aaa6-565d03a081de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:19:18.291463 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.291410    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b36bc16-de79-4aae-aaa6-565d03a081de" (UID: "9b36bc16-de79-4aae-aaa6-565d03a081de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:19:18.291744 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.291714    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b36bc16-de79-4aae-aaa6-565d03a081de-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b36bc16-de79-4aae-aaa6-565d03a081de" (UID: "9b36bc16-de79-4aae-aaa6-565d03a081de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 04:19:18.291913 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.291785    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b36bc16-de79-4aae-aaa6-565d03a081de" (UID: "9b36bc16-de79-4aae-aaa6-565d03a081de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:19:18.291913 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.291853    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-token\") pod \"9b36bc16-de79-4aae-aaa6-565d03a081de\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") "
Jul 09 04:19:18.292035 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.291901    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b36bc16-de79-4aae-aaa6-565d03a081de" (UID: "9b36bc16-de79-4aae-aaa6-565d03a081de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:19:18.292035 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.291958    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-podinfo\") pod \"9b36bc16-de79-4aae-aaa6-565d03a081de\" (UID: \"9b36bc16-de79-4aae-aaa6-565d03a081de\") "
Jul 09 04:19:18.292263 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.292245    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-credential-socket\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:18.292324 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.292271    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-data\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:18.292324 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.292286    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-certs\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:18.292324 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.292300    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b36bc16-de79-4aae-aaa6-565d03a081de-istiod-ca-cert\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:18.292324 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.292313    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-workload-socket\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:18.293361 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.293331    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b36bc16-de79-4aae-aaa6-565d03a081de" (UID: "9b36bc16-de79-4aae-aaa6-565d03a081de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 04:19:18.293651 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.293630    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-kube-api-access-ssmct" (OuterVolumeSpecName: "kube-api-access-ssmct") pod "9b36bc16-de79-4aae-aaa6-565d03a081de" (UID: "9b36bc16-de79-4aae-aaa6-565d03a081de"). InnerVolumeSpecName "kube-api-access-ssmct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:19:18.293857 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.293840    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b36bc16-de79-4aae-aaa6-565d03a081de" (UID: "9b36bc16-de79-4aae-aaa6-565d03a081de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 04:19:18.293991 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.293975    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b36bc16-de79-4aae-aaa6-565d03a081de" (UID: "9b36bc16-de79-4aae-aaa6-565d03a081de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 04:19:18.393345 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.393306    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-token\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:18.393345 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.393339    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-podinfo\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:18.393345 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.393348    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b36bc16-de79-4aae-aaa6-565d03a081de-istio-envoy\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:18.393345 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:18.393357    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssmct\" (UniqueName: \"kubernetes.io/projected/9b36bc16-de79-4aae-aaa6-565d03a081de-kube-api-access-ssmct\") on node \"ip-10-0-134-164.ec2.internal\" DevicePath \"\""
Jul 09 04:19:19.170669 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:19.170635    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"
Jul 09 04:19:19.171088 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:19.170662    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg" event={"ID":"9b36bc16-de79-4aae-aaa6-565d03a081de","Type":"ContainerDied","Data":"3af5ea54ae06466ca7bade075598439efc8bf9a3756f4f532958334a4b2ffdbe"}
Jul 09 04:19:19.171088 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:19.170725    2582 scope.go:117] "RemoveContainer" containerID="84f4750ea78f41ef20cf624dd05054aed8736707d85b5d6fea850a754123f3f6"
Jul 09 04:19:19.193448 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:19.193408    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"]
Jul 09 04:19:19.197150 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:19.197118    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a3bdc1-openshift-default-564ff7d75-ctgbg"]
Jul 09 04:19:19.653582 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:19.653546    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:19.713908 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:19.713859    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" path="/var/lib/kubelet/pods/9b36bc16-de79-4aae-aaa6-565d03a081de/volumes"
Jul 09 04:19:19.753213 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:19.753172    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:19.922829 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:19.922738    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:26.002748 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:26.002720    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645c4d5897-b5kjp_2b175cce-af26-480d-aa42-cac582954955/copy-manifests/0.log"
Jul 09 04:19:26.024093 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:26.024059    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645c4d5897-b5kjp_2b175cce-af26-480d-aa42-cac582954955/manager/0.log"
Jul 09 04:19:27.087750 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.087719    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5_e5894dae-05db-4a52-b5e4-d66e7e98b300/pull/0.log"
Jul 09 04:19:27.094153 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.094129    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5_e5894dae-05db-4a52-b5e4-d66e7e98b300/extract/0.log"
Jul 09 04:19:27.100116 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.100093    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5_e5894dae-05db-4a52-b5e4-d66e7e98b300/util/0.log"
Jul 09 04:19:27.208583 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.208555    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf_f2d469b7-157d-42c5-a735-187c703c2f4e/util/0.log"
Jul 09 04:19:27.214965 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.214938    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf_f2d469b7-157d-42c5-a735-187c703c2f4e/pull/0.log"
Jul 09 04:19:27.221974 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.221945    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf_f2d469b7-157d-42c5-a735-187c703c2f4e/extract/0.log"
Jul 09 04:19:27.334087 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.334060    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp_87c2dbea-9078-4ff6-afde-9c8b46b1dd20/util/0.log"
Jul 09 04:19:27.340387 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.340320    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp_87c2dbea-9078-4ff6-afde-9c8b46b1dd20/pull/0.log"
Jul 09 04:19:27.346372 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.346351    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp_87c2dbea-9078-4ff6-afde-9c8b46b1dd20/extract/0.log"
Jul 09 04:19:27.468534 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.468502    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6_d2c8bfbc-6e27-4051-b42d-c9679264a73d/util/0.log"
Jul 09 04:19:27.474521 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.474492    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6_d2c8bfbc-6e27-4051-b42d-c9679264a73d/pull/0.log"
Jul 09 04:19:27.480068 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:27.480046    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6_d2c8bfbc-6e27-4051-b42d-c9679264a73d/extract/0.log"
Jul 09 04:19:28.209964 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:28.209926    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-cbh2z_362aa580-7601-490e-8a64-b444c4019e02/manager/0.log"
Jul 09 04:19:28.968573 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:28.968542    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-59nvr_7b285065-affa-4582-9997-e5ac1a4ac3e9/discovery/0.log"
Jul 09 04:19:30.007662 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:30.007628    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-268jr_cfb87f88-2238-472e-9a92-1ac5f1f70e57/main/0.log"
Jul 09 04:19:30.245780 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:30.245744    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jff25_62b89f9f-c9e9-420f-9cec-c33094408036/main/0.log"
Jul 09 04:19:34.784235 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784192    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b9skj/must-gather-zm8sw"]
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784496    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27906d5c-57dd-4aa6-98dc-13b017ebf669" containerName="maas-api"
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784509    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="27906d5c-57dd-4aa6-98dc-13b017ebf669" containerName="maas-api"
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784518    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy"
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784525    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy"
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784543    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da8ef171-235a-4695-8e40-e00dad8226e7" containerName="maas-api"
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784552    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="da8ef171-235a-4695-8e40-e00dad8226e7" containerName="maas-api"
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784566    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="042df7af-eaf9-435b-9c72-75474ab8029d" containerName="cleanup"
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784576    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="042df7af-eaf9-435b-9c72-75474ab8029d" containerName="cleanup"
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784637    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy"
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784649    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy"
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784659    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="042df7af-eaf9-435b-9c72-75474ab8029d" containerName="cleanup"
Jul 09 04:19:34.784660 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784666    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="da8ef171-235a-4695-8e40-e00dad8226e7" containerName="maas-api"
Jul 09 04:19:34.785135 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784677    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="27906d5c-57dd-4aa6-98dc-13b017ebf669" containerName="maas-api"
Jul 09 04:19:34.785135 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784754    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy"
Jul 09 04:19:34.785135 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.784766    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b36bc16-de79-4aae-aaa6-565d03a081de" containerName="istio-proxy"
Jul 09 04:19:34.789235 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.789218    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b9skj/must-gather-zm8sw"
Jul 09 04:19:34.792202 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.792178    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b9skj\"/\"kube-root-ca.crt\""
Jul 09 04:19:34.792381 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.792228    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b9skj\"/\"openshift-service-ca.crt\""
Jul 09 04:19:34.792381 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.792258    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b9skj\"/\"default-dockercfg-2zmlk\""
Jul 09 04:19:34.797489 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.797072    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b9skj/must-gather-zm8sw"]
Jul 09 04:19:34.801772 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.801736    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:34.858937 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.858845    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log" limit=1024
Jul 09 04:19:34.943573 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.943540    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkmtt\" (UniqueName: \"kubernetes.io/projected/8d1fb0f4-d279-4ee8-a42c-33d6f3e18444-kube-api-access-fkmtt\") pod \"must-gather-zm8sw\" (UID: \"8d1fb0f4-d279-4ee8-a42c-33d6f3e18444\") " pod="openshift-must-gather-b9skj/must-gather-zm8sw"
Jul 09 04:19:34.943746 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:34.943589    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8d1fb0f4-d279-4ee8-a42c-33d6f3e18444-must-gather-output\") pod \"must-gather-zm8sw\" (UID: \"8d1fb0f4-d279-4ee8-a42c-33d6f3e18444\") " pod="openshift-must-gather-b9skj/must-gather-zm8sw"
Jul 09 04:19:35.044065 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:35.043968    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8d1fb0f4-d279-4ee8-a42c-33d6f3e18444-must-gather-output\") pod \"must-gather-zm8sw\" (UID: \"8d1fb0f4-d279-4ee8-a42c-33d6f3e18444\") " pod="openshift-must-gather-b9skj/must-gather-zm8sw"
Jul 09 04:19:35.044222 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:35.044089    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkmtt\" (UniqueName: \"kubernetes.io/projected/8d1fb0f4-d279-4ee8-a42c-33d6f3e18444-kube-api-access-fkmtt\") pod \"must-gather-zm8sw\" (UID: \"8d1fb0f4-d279-4ee8-a42c-33d6f3e18444\") " pod="openshift-must-gather-b9skj/must-gather-zm8sw"
Jul 09 04:19:35.044395 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:35.044374    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8d1fb0f4-d279-4ee8-a42c-33d6f3e18444-must-gather-output\") pod \"must-gather-zm8sw\" (UID: \"8d1fb0f4-d279-4ee8-a42c-33d6f3e18444\") " pod="openshift-must-gather-b9skj/must-gather-zm8sw"
Jul 09 04:19:35.052458 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:35.052428    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkmtt\" (UniqueName: \"kubernetes.io/projected/8d1fb0f4-d279-4ee8-a42c-33d6f3e18444-kube-api-access-fkmtt\") pod \"must-gather-zm8sw\" (UID: \"8d1fb0f4-d279-4ee8-a42c-33d6f3e18444\") " pod="openshift-must-gather-b9skj/must-gather-zm8sw"
Jul 09 04:19:35.100529 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:35.100474    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b9skj/must-gather-zm8sw"
Jul 09 04:19:35.229453 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:35.229424    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b9skj/must-gather-zm8sw"]
Jul 09 04:19:35.231340 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:19:35.231306    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d1fb0f4_d279_4ee8_a42c_33d6f3e18444.slice/crio-1b71cb397b5f03fb6aaa7abd0d639fad5306dab508992e8ff5d86f275aed7a4a WatchSource:0}: Error finding container 1b71cb397b5f03fb6aaa7abd0d639fad5306dab508992e8ff5d86f275aed7a4a: Status 404 returned error can't find the container with id 1b71cb397b5f03fb6aaa7abd0d639fad5306dab508992e8ff5d86f275aed7a4a
Jul 09 04:19:35.233201 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:35.233184    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 04:19:35.252023 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:35.251988    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9skj/must-gather-zm8sw" event={"ID":"8d1fb0f4-d279-4ee8-a42c-33d6f3e18444","Type":"ContainerStarted","Data":"1b71cb397b5f03fb6aaa7abd0d639fad5306dab508992e8ff5d86f275aed7a4a"}
Jul 09 04:19:37.276007 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:37.275965    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9skj/must-gather-zm8sw" event={"ID":"8d1fb0f4-d279-4ee8-a42c-33d6f3e18444","Type":"ContainerStarted","Data":"aeb98b13219ff9cb157a2d2370d0310a0e157ac95a3eb36ed4aeecc2793f5914"}
Jul 09 04:19:37.276487 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:37.276016    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9skj/must-gather-zm8sw" event={"ID":"8d1fb0f4-d279-4ee8-a42c-33d6f3e18444","Type":"ContainerStarted","Data":"64ec0cfc72def320aa0f69e1678bc4715272a55d3b13dc7518b6f2699a8d0fe7"}
Jul 09 04:19:37.294493 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:37.294426    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b9skj/must-gather-zm8sw" podStartSLOduration=2.226011024 podStartE2EDuration="3.294402485s" totalImagesPullingTime="1.068391461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:19:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:19:37.290832696 +0000 UTC m=+2194.110088483" watchObservedRunningTime="2026-07-09 04:19:37.294402485 +0000 UTC m=+2194.113658263"
Jul 09 04:19:37.903694 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:37.903659    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pw8dq_56157167-4b57-4f59-9f0b-8925fea4b4b3/global-pull-secret-syncer/0.log"
Jul 09 04:19:37.966111 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:37.966073    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4bcnc_2a8cd1aa-e1ab-43b7-b1d5-0c91cd7b805f/konnectivity-agent/0.log"
Jul 09 04:19:38.054928 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:38.054893    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-164.ec2.internal_3d922faf733bce56c232f28d751a7e10/haproxy/0.log"
Jul 09 04:19:43.171446 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.171411    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5_e5894dae-05db-4a52-b5e4-d66e7e98b300/extract/0.log"
Jul 09 04:19:43.207019 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.206985    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5_e5894dae-05db-4a52-b5e4-d66e7e98b300/util/0.log"
Jul 09 04:19:43.232615 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.232589    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759spwg5_e5894dae-05db-4a52-b5e4-d66e7e98b300/pull/0.log"
Jul 09 04:19:43.270983 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.270936    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf_f2d469b7-157d-42c5-a735-187c703c2f4e/extract/0.log"
Jul 09 04:19:43.294694 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.294668    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf_f2d469b7-157d-42c5-a735-187c703c2f4e/util/0.log"
Jul 09 04:19:43.319094 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.319063    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vv5jf_f2d469b7-157d-42c5-a735-187c703c2f4e/pull/0.log"
Jul 09 04:19:43.348018 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.347981    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp_87c2dbea-9078-4ff6-afde-9c8b46b1dd20/extract/0.log"
Jul 09 04:19:43.373067 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.372982    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp_87c2dbea-9078-4ff6-afde-9c8b46b1dd20/util/0.log"
Jul 09 04:19:43.396511 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.396475    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vqqp_87c2dbea-9078-4ff6-afde-9c8b46b1dd20/pull/0.log"
Jul 09 04:19:43.425226 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.425085    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6_d2c8bfbc-6e27-4051-b42d-c9679264a73d/extract/0.log"
Jul 09 04:19:43.446615 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.446583    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6_d2c8bfbc-6e27-4051-b42d-c9679264a73d/util/0.log"
Jul 09 04:19:43.469839 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:43.469808    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nn6r6_d2c8bfbc-6e27-4051-b42d-c9679264a73d/pull/0.log"
Jul 09 04:19:44.003711 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:44.003645    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-cbh2z_362aa580-7601-490e-8a64-b444c4019e02/manager/0.log"
Jul 09 04:19:46.248100 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:46.248058    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jcnfn_f76f7924-686f-446e-905b-b343df740a64/node-exporter/0.log"
Jul 09 04:19:46.269761 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:46.269723    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jcnfn_f76f7924-686f-446e-905b-b343df740a64/kube-rbac-proxy/0.log"
Jul 09 04:19:46.294610 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:46.294581    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jcnfn_f76f7924-686f-446e-905b-b343df740a64/init-textfile/0.log"
Jul 09 04:19:46.717133 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:46.717096    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-869fd9cd76-mpqxt_5042b609-c946-4f78-bb43-b34d20a70c57/telemeter-client/0.log"
Jul 09 04:19:46.738714 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:46.738671    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-869fd9cd76-mpqxt_5042b609-c946-4f78-bb43-b34d20a70c57/reload/0.log"
Jul 09 04:19:46.759787 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:46.759758    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-869fd9cd76-mpqxt_5042b609-c946-4f78-bb43-b34d20a70c57/kube-rbac-proxy/0.log"
Jul 09 04:19:48.193622 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.193582    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"]
Jul 09 04:19:48.198894 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.198840    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.208000 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.207959    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"]
Jul 09 04:19:48.390340 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.390290    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-proc\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.390340 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.390344    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-lib-modules\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.390672 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.390395    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-sys\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.390672 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.390438    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-podres\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.390672 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.390480    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccnnr\" (UniqueName: \"kubernetes.io/projected/b198373d-3e96-4b05-904e-7561c4ae2b19-kube-api-access-ccnnr\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.491485 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.491362    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-proc\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.491485 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.491423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-lib-modules\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.491485 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.491474    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-sys\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.491792 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.491511    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-proc\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.491792 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.491519    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-podres\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.491792 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.491603    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccnnr\" (UniqueName: \"kubernetes.io/projected/b198373d-3e96-4b05-904e-7561c4ae2b19-kube-api-access-ccnnr\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.491792 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.491639    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-podres\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.491792 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.491657    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-sys\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.491792 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.491722    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b198373d-3e96-4b05-904e-7561c4ae2b19-lib-modules\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.500321 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.500276    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccnnr\" (UniqueName: \"kubernetes.io/projected/b198373d-3e96-4b05-904e-7561c4ae2b19-kube-api-access-ccnnr\") pod \"perf-node-gather-daemonset-ttnhq\" (UID: \"b198373d-3e96-4b05-904e-7561c4ae2b19\") " pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.514975 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.514930    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:48.709225 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:48.709185    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"]
Jul 09 04:19:48.713499 ip-10-0-134-164 kubenswrapper[2582]: W0709 04:19:48.713460    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb198373d_3e96_4b05_904e_7561c4ae2b19.slice/crio-28038dcc0423908079bea63c12acda4607256ddd7ba7ce9a5ea0bc126c8c07c3 WatchSource:0}: Error finding container 28038dcc0423908079bea63c12acda4607256ddd7ba7ce9a5ea0bc126c8c07c3: Status 404 returned error can't find the container with id 28038dcc0423908079bea63c12acda4607256ddd7ba7ce9a5ea0bc126c8c07c3
Jul 09 04:19:49.390088 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:49.390045    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq" event={"ID":"b198373d-3e96-4b05-904e-7561c4ae2b19","Type":"ContainerStarted","Data":"c7a97f3d8a69ef05cd7a6e25aab4e0cf4d9b9c9cec9eff9f5bd04936cc1e7631"}
Jul 09 04:19:49.390695 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:49.390673    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:49.390842 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:49.390820    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq" event={"ID":"b198373d-3e96-4b05-904e-7561c4ae2b19","Type":"ContainerStarted","Data":"28038dcc0423908079bea63c12acda4607256ddd7ba7ce9a5ea0bc126c8c07c3"}
Jul 09 04:19:50.972379 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:50.972346    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4gjlw_b9f541c8-fdea-4bf7-b3f2-61ae2262afc2/dns/0.log"
Jul 09 04:19:50.994855 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:50.994817    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4gjlw_b9f541c8-fdea-4bf7-b3f2-61ae2262afc2/kube-rbac-proxy/0.log"
Jul 09 04:19:51.115681 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:51.115650    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2wsms_fd5a5f67-7cb7-4196-b7ef-8f3227eed070/dns-node-resolver/0.log"
Jul 09 04:19:51.763235 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:51.763204    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x655t_c2af28c4-c7f0-4545-9b84-04f7ec3c9533/node-ca/0.log"
Jul 09 04:19:52.909809 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:52.909769    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-59nvr_7b285065-affa-4582-9997-e5ac1a4ac3e9/discovery/0.log"
Jul 09 04:19:53.879064 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:53.879031    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xv4td_478e4ae2-fc93-4443-93f2-83f1208cb291/serve-healthcheck-canary/0.log"
Jul 09 04:19:54.601873 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:54.601825    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n29bj_fe660e68-2687-427e-94c8-2d8fc456a42c/kube-rbac-proxy/0.log"
Jul 09 04:19:54.626905 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:54.626873    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n29bj_fe660e68-2687-427e-94c8-2d8fc456a42c/exporter/0.log"
Jul 09 04:19:54.651489 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:54.651458    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n29bj_fe660e68-2687-427e-94c8-2d8fc456a42c/extractor/0.log"
Jul 09 04:19:55.406315 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:55.406286    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq"
Jul 09 04:19:55.426671 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:55.426616    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b9skj/perf-node-gather-daemonset-ttnhq" podStartSLOduration=7.426596986 podStartE2EDuration="7.426596986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:19:49.409585952 +0000 UTC m=+2206.228841745" watchObservedRunningTime="2026-07-09 04:19:55.426596986 +0000 UTC m=+2212.245852763"
Jul 09 04:19:57.275527 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:57.275443    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645c4d5897-b5kjp_2b175cce-af26-480d-aa42-cac582954955/manager/0.log"
Jul 09 04:19:57.296280 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:57.296252    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645c4d5897-b5kjp_2b175cce-af26-480d-aa42-cac582954955/copy-manifests/0.log"
Jul 09 04:19:58.826051 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:58.826014    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7f5f9c4b6d-x4tgr_62a1507d-4169-4cc1-a312-088304e18bac/manager/0.log"
Jul 09 04:19:58.851289 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:58.851250    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-gmdqq_334257fc-37cd-4f40-a3ef-6bcb256a8f72/openshift-lws-operator/0.log"
Jul 09 04:19:59.239895 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:19:59.239770    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-59nvr_7b285065-affa-4582-9997-e5ac1a4ac3e9/discovery/0.log"
Jul 09 04:20:00.796213 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:20:00.796156    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-59nvr_7b285065-affa-4582-9997-e5ac1a4ac3e9/discovery/0.log"
Jul 09 04:20:05.396790 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:20:05.396751    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-942xr_ee8e2cd2-4854-4d43-be1a-9a0c1bf3ce72/kube-multus/0.log"
Jul 09 04:20:05.422540 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:20:05.422508    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24dwk_be397f8d-fd42-44ea-b444-3530fe24606b/kube-multus-additional-cni-plugins/0.log"
Jul 09 04:20:05.443288 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:20:05.443261    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24dwk_be397f8d-fd42-44ea-b444-3530fe24606b/egress-router-binary-copy/0.log"
Jul 09 04:20:05.464540 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:20:05.464511    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24dwk_be397f8d-fd42-44ea-b444-3530fe24606b/cni-plugins/0.log"
Jul 09 04:20:05.486716 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:20:05.486682    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24dwk_be397f8d-fd42-44ea-b444-3530fe24606b/bond-cni-plugin/0.log"
Jul 09 04:20:05.508904 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:20:05.508859    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24dwk_be397f8d-fd42-44ea-b444-3530fe24606b/routeoverride-cni/0.log"
Jul 09 04:20:05.534652 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:20:05.534618    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24dwk_be397f8d-fd42-44ea-b444-3530fe24606b/whereabouts-cni-bincopy/0.log"
Jul 09 04:20:05.561229 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:20:05.561198    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24dwk_be397f8d-fd42-44ea-b444-3530fe24606b/whereabouts-cni/0.log"
Jul 09 04:20:06.008186 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:20:06.007962    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-czw4n_15c1454e-931d-4ab2-8bcc-20db5808084e/network-metrics-daemon/0.log"
Jul 09 04:20:06.028548 ip-10-0-134-164 kubenswrapper[2582]: I0709 04:20:06.028515    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-czw4n_15c1454e-931d-4ab2-8bcc-20db5808084e/kube-rbac-proxy/0.log"