Jul 17 05:39:08.492864 ip-10-0-140-96 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 17 05:39:08.492876 ip-10-0-140-96 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 17 05:39:08.492882 ip-10-0-140-96 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 17 05:39:08.493094 ip-10-0-140-96 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 17 05:39:19.889175 ip-10-0-140-96 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 17 05:39:19.889191 ip-10-0-140-96 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ef5fb6078fb7432696782cb9282f97ef --
Jul 17 05:41:17.452743 ip-10-0-140-96 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 05:41:17.815057 ip-10-0-140-96 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 05:41:17.815057 ip-10-0-140-96 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 05:41:17.815057 ip-10-0-140-96 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 05:41:17.815057 ip-10-0-140-96 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 05:41:17.815057 ip-10-0-140-96 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 05:41:17.816064 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.815979    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 05:41:17.822115 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822099    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:41:17.822115 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822114    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822118    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822121    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822124    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822127    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822130    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822132    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822135    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822137    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822139    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822142    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822144    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822147    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822149    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822151    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822154    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822156    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822159    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822162    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822164    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:41:17.822181 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822166    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822169    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822171    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822173    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822176    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822178    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822188    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822192    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822194    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822197    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822199    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822202    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822205    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822208    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822212    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822214    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822217    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822219    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822222    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:41:17.822651 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822224    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822226    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822229    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822231    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822234    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822236    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822238    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822241    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822243    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822245    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822248    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822250    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822253    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822257    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822261    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822264    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822267    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822270    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822273    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822275    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:41:17.823103 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822278    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822281    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822285    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822288    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822290    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822293    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822295    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822297    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822300    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822302    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822305    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822307    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822310    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822312    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822315    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822317    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822321    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822323    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822326    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:41:17.823628 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822328    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822331    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822333    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822339    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822341    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822344    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822346    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822349    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822352    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822354    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822357    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822359    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822362    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822365    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822367    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822370    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822373    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822375    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822378    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.822380    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:41:17.824063 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823056    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823063    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823067    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823070    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823073    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823075    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823078    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823080    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823082    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823085    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823087    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823091    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823094    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823096    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823099    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823103    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823105    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823108    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823110    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823113    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:41:17.824551 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823115    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823119    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823123    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823126    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823129    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823131    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823134    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823137    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823139    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823142    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823145    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823147    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823150    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823152    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823155    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823157    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823159    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823162    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823164    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823167    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:41:17.825029 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823170    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823172    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823174    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823177    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823180    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823184    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823187    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823191    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823194    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823197    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823200    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823203    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823205    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823208    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823210    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823212    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823215    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823217    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823220    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:41:17.825503 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823222    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823225    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823227    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823231    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823233    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823236    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823238    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823241    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823243    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823245    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823248    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823250    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823253    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823255    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823258    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823260    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823262    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823265    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823267    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823270    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:41:17.825942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823272    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823275    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823277    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823280    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823282    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823285    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823287    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823289    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823292    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823294    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823296    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823299    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823301    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823304    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823306    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823309    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823311    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823313    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823315    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:41:17.826408 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823318    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823708    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823714    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823716    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823719    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823722    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823724    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823726    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823729    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823732    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823734    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823738    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823740    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823743    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823745    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823748    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823750    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823753    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823756    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823758    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:41:17.826868 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823760    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823763    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823765    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823768    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823770    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823772    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823775    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823778    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823780    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823782    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823785    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823787    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823791    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823794    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823797    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823799    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823801    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823806    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823808    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:41:17.827347 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823810    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823813    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823815    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823818    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823821    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823824    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823826    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823829    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823832    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823836    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823839    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823842    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823846    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823848    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823851    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823854    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823857    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823859    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823862    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:41:17.827792 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823865    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823867    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823869    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823872    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823874    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823876    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823879    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823881    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823884    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823887    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823891    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823894    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823897    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823899    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823902    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823905    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823907    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823910    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823913    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823916    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:41:17.828266 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823919    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823922    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823925    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823927    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823931    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823933    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823936    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823939    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823941    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823944    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823946    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823949    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823951    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823954    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823956    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823958    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823961    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823963    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823966    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823968    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:41:17.828749 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823970    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.823973    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824043    2558 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824051    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824058    2558 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824065    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824072    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824077    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824083    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824087    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824090    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824094    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824098    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824101    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824104    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824107    2558 flags.go:64] FLAG: --cgroup-root=""
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824110    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824113    2558 flags.go:64] FLAG: --client-ca-file=""
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824115    2558 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824118    2558 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824122    2558 flags.go:64] FLAG: --cluster-domain=""
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824124    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824127    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824130    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 05:41:17.829216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824133    2558 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824137    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824139    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824142    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824145    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824148    2558 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824150    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824153    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824156    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824158    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824162    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824166    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824168    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824171    2558 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824174    2558 flags.go:64] FLAG: --enable-server="true"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824176    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824181    2558 flags.go:64] FLAG: --event-burst="100"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824183    2558 flags.go:64] FLAG: --event-qps="50"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824186    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824190    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824192    2558 flags.go:64] FLAG: --eviction-hard=""
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824196    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824199    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824202    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824205    2558 flags.go:64] FLAG: --eviction-soft=""
Jul 17 05:41:17.829781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824207    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824210    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824212    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824215    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824218    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824220    2558 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824223    2558 flags.go:64] FLAG: --feature-gates=""
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824226    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824229    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824231    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824234    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824237    2558 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824240    2558 flags.go:64] FLAG: --help="false"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824242    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-140-96.ec2.internal"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824245    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824248    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824250    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824253    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824258    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824261    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824263    2558 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824266    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824269    2558 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 05:41:17.830365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824271    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824274    2558 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824277    2558 flags.go:64] FLAG: --kube-reserved=""
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824282    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824284    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824287    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824290    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824293    2558 flags.go:64] FLAG: --lock-file=""
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824296    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824298    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824301    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824306    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824309    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824311    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824314    2558 flags.go:64] FLAG: --logging-format="text"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824316    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824319    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824322    2558 flags.go:64] FLAG: --manifest-url=""
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824324    2558 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824328    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824331    2558 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824335    2558 flags.go:64] FLAG: --max-pods="110"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824338    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824340    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824343    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 05:41:17.831193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824346    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824348    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824353    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824355    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824365    2558 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824367    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824370    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824373    2558 flags.go:64] FLAG: --pod-cidr=""
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824376    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824381    2558 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824383    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824387    2558 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824390    2558 flags.go:64] FLAG: --port="10250"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824393    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824396    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05223ebbed0c107a8"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824399    2558 flags.go:64] FLAG: --qos-reserved=""
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824402    2558 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824405    2558 flags.go:64] FLAG: --register-node="true"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824407    2558 flags.go:64] FLAG: --register-with-taints=""
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824411    2558 flags.go:64] FLAG: --registry-burst="10"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824413    2558 flags.go:64] FLAG: --registry-qps="5"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824416    2558 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824418    2558 flags.go:64] FLAG: --reserved-memory=""
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824421    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824425    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 05:41:17.832261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824427    2558 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824430    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824432    2558 flags.go:64] FLAG: --runonce="false"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824435    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824438    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824440    2558 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824443    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824446    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824449    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824453    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824456    2558 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824458    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824461    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824463    2558 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824466    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824483    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824486    2558 flags.go:64] FLAG: --system-cgroups=""
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824489    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824496    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824499    2558 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824501    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824505    2558 flags.go:64] FLAG: --tls-min-version=""
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824508    2558 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824511    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824514    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 05:41:17.833294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824516    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824519    2558 flags.go:64] FLAG: --v="2"
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824523    2558 flags.go:64] FLAG: --version="false"
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824526    2558 flags.go:64] FLAG: --vmodule=""
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824530    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.824533    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824646    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824650    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824652    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824656    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824658    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824660    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824663    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824666    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824668    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824670    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824674    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824677    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824679    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824682    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824684    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:41:17.834344 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824687    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824689    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824692    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824696    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824701    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824703    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824706    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824709    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824712    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824715    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824718    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824720    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824723    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824725    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824728    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824731    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824734    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824738    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824740    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:41:17.834943 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824743    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824746    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824748    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824751    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824754    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824757    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824759    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824762    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824767    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824770    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824773    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824775    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824777    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824780    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824782    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824785    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824787    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824790    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824793    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:41:17.835735 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824795    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824798    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824801    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824803    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824806    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824808    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824811    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824813    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824816    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824818    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824820    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824823    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824825    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824827    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824830    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824832    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824835    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824837    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824840    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824842    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:41:17.836428 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824844    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824848    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824851    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824854    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824856    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824859    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824861    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824863    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824866    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824868    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824871    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824874    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824876    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824879    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824881    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824883    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824886    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824889    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824892    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824895    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:41:17.837046 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824897    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824899    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824902    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824904    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824907    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.824909    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.825390    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.832860    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.832880    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.832902    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.832911    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.832992    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833000    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833005    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833009    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:41:17.837632 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833014    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833018    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833021    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833026    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833030    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833034    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833039    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833043    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833047    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833051    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833055    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833059    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833063    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833068    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833072    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833076    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833080    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833084    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833088    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833092    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:41:17.838001 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833096    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833100    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833104    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833108    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833112    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833119    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833123    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833127    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833131    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833135    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833140    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833144    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833148    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833152    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833156    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833161    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833165    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833169    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833173    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833177    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:41:17.838515 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833181    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833185    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833189    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833193    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833197    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833201    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833206    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833210    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833213    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833217    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833221    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833225    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833229    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833233    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833237    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833242    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833245    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833251    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833256    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:41:17.838988 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833262    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833268    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833273    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833278    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833282    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833287    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833291    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833295    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833299    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833302    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833306    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833310    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833314    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833317    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833323    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833329    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833333    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833338    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833342    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:41:17.839438 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833346    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833351    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833355    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833359    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833363    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833367    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833371    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833375    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833379    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833384    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833388    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833392    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833396    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833401    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833408    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833413    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:41:17.839897 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833418    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.833426    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833658    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833667    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833672    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833677    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833681    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833687    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833691    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833695    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833699    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833703    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833707    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833713    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833720    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 05:41:17.840274 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833725    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833729    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833733    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833738    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833742    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833746    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833750    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833754    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833758    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833762    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833767    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833771    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833775    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833780    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833784    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833789    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833793    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833797    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833801    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 05:41:17.840647 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833805    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833812    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833817    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833822    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833826    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833830    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833835    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833839    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833843    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833847    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833851    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833855    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833859    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833863    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833867    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833871    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833875    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833880    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833884    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833888    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 05:41:17.841098 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833892    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833896    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833900    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833904    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833908    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833912    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833916    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833920    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833924    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833930    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833934    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833939    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833942    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833946    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833950    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833954    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833958    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833962    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833966    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 05:41:17.841622 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833970    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833976    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833982    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833986    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833990    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833995    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.833999    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834003    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834008    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834012    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834016    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834019    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834023    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834027    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834031    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834036    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834040    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834043    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834047    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834051    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 05:41:17.842065 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834055    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834059    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834063    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834068    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834072    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834076    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834080    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:17.834084    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.834091    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.834237    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.836849    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.838285    2558 server.go:1019] "Starting client certificate rotation"
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.838398    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 05:41:17.842889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.838440    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 05:41:17.843379 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.841318    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 05:41:17.844648 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.844628    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 05:41:17.853368 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.853352    2558 log.go:25] "Validated CRI v1 runtime API"
Jul 17 05:41:17.855237 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.855219    2558 log.go:25] "Validated CRI v1 image API"
Jul 17 05:41:17.856067 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.856049    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 05:41:17.858257 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.858232    2558 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9583b9d6-5b9f-46e9-87cc-8565b276d2cb:/dev/nvme0n1p4 f782c153-8001-4614-bb27-df32dc120ea0:/dev/nvme0n1p3]
Jul 17 05:41:17.858315 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.858258    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 05:41:17.863144 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.863029    2558 manager.go:216] Machine: {Timestamp:2026-07-17 05:41:17.861763923 +0000 UTC m=+0.303702488 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3199222 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec284607b2407994fc4c4333f11a58b2 SystemUUID:ec284607-b240-7994-fc4c-4333f11a58b2 BootID:ef5fb607-8fb7-4326-9678-2cb9282f97ef Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:56:c8:8b:7d:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:56:c8:8b:7d:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:8c:3d:a2:c3:2f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 05:41:17.863144 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.863142    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 05:41:17.863233 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.863216    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 05:41:17.866218 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.866195    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 05:41:17.866346 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.866222    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-96.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 05:41:17.866999 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.866989    2558 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 05:41:17.867035 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.867002    2558 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 05:41:17.867035 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.867009    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 05:41:17.867125 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.867110    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 05:41:17.867822 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.867812    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 05:41:17.868998 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.868989    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 05:41:17.871889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.871878    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 05:41:17.871994 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.871985    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 05:41:17.872030 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.872014    2558 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 05:41:17.872030 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.872024    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 05:41:17.872100 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.872035    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 05:41:17.872100 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.872045    2558 kubelet.go:398] "Adding apiserver pod source"
Jul 17 05:41:17.872100 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.872053    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 05:41:17.872368 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.872355    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 05:41:17.872400 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.872376    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 05:41:17.874583 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.874566    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 05:41:17.876327 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.876313    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 05:41:17.876379 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.876343    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 05:41:17.877370 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877357    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 05:41:17.877421 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877374    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 05:41:17.877421 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877381    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 05:41:17.877421 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877387    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 05:41:17.877421 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877397    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 05:41:17.877421 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877405    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 05:41:17.877421 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877413    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 05:41:17.877421 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877418    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 05:41:17.877675 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877425    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 05:41:17.877675 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877431    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 05:41:17.877675 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877439    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 05:41:17.877675 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877444    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 05:41:17.877675 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877485    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 05:41:17.877675 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.877493    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 05:41:17.880649 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.880635    2558 server.go:1268] "Started kubelet"
Jul 17 05:41:17.880800 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.880764    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 05:41:17.880854 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.880793    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 05:41:17.880854 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.880850    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 05:41:17.886039 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.886012    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 05:41:17.886247 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.886228    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-96.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 05:41:17.886576 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:17.886495    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-96.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 05:41:17.886576 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:17.886531    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 05:41:17.887107 ip-10-0-140-96 systemd[1]: Started Kubernetes Kubelet.
Jul 17 05:41:17.888507 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.888380    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hr8mk"
Jul 17 05:41:17.888507 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.888311    2558 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 05:41:17.889282 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:17.889259    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 05:41:17.891341 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:17.890291    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-96.ec2.internal.18c2fd45a091ad20  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-96.ec2.internal,UID:ip-10-0-140-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-96.ec2.internal,},FirstTimestamp:2026-07-17 05:41:17.88061008 +0000 UTC m=+0.322548692,LastTimestamp:2026-07-17 05:41:17.88061008 +0000 UTC m=+0.322548692,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-96.ec2.internal,}"
Jul 17 05:41:17.896021 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.896000    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 05:41:17.896091 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.896003    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 05:41:17.896219 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:17.896201    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:17.896605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.896573    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hr8mk"
Jul 17 05:41:17.896716 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.896701    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 05:41:17.896804 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.896785    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 05:41:17.896919 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.896701    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 05:41:17.896919 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.896907    2558 factory.go:55] Registering systemd factory
Jul 17 05:41:17.896919 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.896919    2558 factory.go:223] Registration of the systemd container factory successfully
Jul 17 05:41:17.897056 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.896931    2558 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 05:41:17.897056 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.896938    2558 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 05:41:17.897309 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.897294    2558 factory.go:153] Registering CRI-O factory
Jul 17 05:41:17.897367 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.897312    2558 factory.go:223] Registration of the crio container factory successfully
Jul 17 05:41:17.897367 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.897362    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 05:41:17.897436 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.897387    2558 factory.go:103] Registering Raw factory
Jul 17 05:41:17.897436 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.897401    2558 manager.go:1196] Started watching for new ooms in manager
Jul 17 05:41:17.898005 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:17.897986    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-96.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 17 05:41:17.898089 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:17.898073    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 17 05:41:17.898198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.898184    2558 manager.go:349] Starting recovery of all containers
Jul 17 05:41:17.908635 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.908358    2558 manager.go:354] Recovery completed
Jul 17 05:41:17.912820 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.912806    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:41:17.914731 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.914711    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:41:17.914827 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.914747    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:41:17.914827 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.914763    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:41:17.915391 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.915375    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 05:41:17.915391 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.915389    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 05:41:17.915532 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.915407    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 05:41:17.915532 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.915436    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:41:17.916153 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.916140    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:41:17.916215 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.916167    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:41:17.916215 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.916177    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:41:17.916673 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.916661    2558 policy_none.go:49] "None policy: Start"
Jul 17 05:41:17.916737 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.916678    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 05:41:17.916737 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.916691    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 05:41:17.917916 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.917903    2558 policy_none.go:47] "Start"
Jul 17 05:41:17.950026 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.950011    2558 manager.go:337] "Starting Device Plugin manager"
Jul 17 05:41:17.950129 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:17.950050    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 05:41:17.950129 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.950060    2558 server.go:87] "Starting device plugin registration server"
Jul 17 05:41:17.950334 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.950321    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 05:41:17.950374 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.950338    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 05:41:17.950429 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.950413    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 05:41:17.950537 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.950525    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 05:41:17.950582 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:17.950538    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 05:41:17.951064 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:17.951035    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 05:41:17.951156 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:17.951098    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.051043 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.051000    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:41:18.052273 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.051970    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:41:18.052273 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.052001    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:41:18.052273 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.052011    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:41:18.052273 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.052037    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.057920 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.057906    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.057974 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.057927    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-96.ec2.internal\": node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.070454 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.070396    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 05:41:18.071668 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.071652    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 05:41:18.071754 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.071693    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 05:41:18.071754 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.071716    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 05:41:18.071863 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.071754    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 05:41:18.071913 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.071871    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.074277 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.074258    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 05:41:18.172648 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.172607    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.172648 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.172603    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-96.ec2.internal"]
Jul 17 05:41:18.172817 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.172708    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:41:18.173815 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.173800    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:41:18.173868 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.173830    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:41:18.173868 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.173843    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:41:18.175169 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.175158    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:41:18.175278 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.175266    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.175333 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.175304    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:41:18.175917 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.175898    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:41:18.176000 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.175926    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:41:18.176000 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.175940    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:41:18.176000 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.175982    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:41:18.176000 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.175997    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:41:18.176139 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.176009    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:41:18.177334 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.177318    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.177424 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.177344    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 05:41:18.178940 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.178925    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 05:41:18.179023 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.178948    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 05:41:18.179023 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.178962    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeHasSufficientPID"
Jul 17 05:41:18.198649 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.198619    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5aa1ec63b2d121d5f010fce817a84937-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal\" (UID: \"5aa1ec63b2d121d5f010fce817a84937\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.198729 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.198653    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5aa1ec63b2d121d5f010fce817a84937-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal\" (UID: \"5aa1ec63b2d121d5f010fce817a84937\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.198729 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.198672    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b714b8177e3d9d42f58f7241552dc9e9-config\") pod \"kube-apiserver-proxy-ip-10-0-140-96.ec2.internal\" (UID: \"b714b8177e3d9d42f58f7241552dc9e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.203030 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.203011    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-96.ec2.internal\" not found" node="ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.207404 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.207383    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-96.ec2.internal\" not found" node="ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.273292 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.273267    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.299726 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.299701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5aa1ec63b2d121d5f010fce817a84937-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal\" (UID: \"5aa1ec63b2d121d5f010fce817a84937\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.299780 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.299730    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5aa1ec63b2d121d5f010fce817a84937-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal\" (UID: \"5aa1ec63b2d121d5f010fce817a84937\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.299780 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.299746    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b714b8177e3d9d42f58f7241552dc9e9-config\") pod \"kube-apiserver-proxy-ip-10-0-140-96.ec2.internal\" (UID: \"b714b8177e3d9d42f58f7241552dc9e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.299842 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.299796    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b714b8177e3d9d42f58f7241552dc9e9-config\") pod \"kube-apiserver-proxy-ip-10-0-140-96.ec2.internal\" (UID: \"b714b8177e3d9d42f58f7241552dc9e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.299842 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.299813    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5aa1ec63b2d121d5f010fce817a84937-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal\" (UID: \"5aa1ec63b2d121d5f010fce817a84937\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.299842 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.299796    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5aa1ec63b2d121d5f010fce817a84937-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal\" (UID: \"5aa1ec63b2d121d5f010fce817a84937\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.374093 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.374039    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.474549 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.474512    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.506096 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.506069    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.510809 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.510791    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:18.574675 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.574645    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.676870 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.676829    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.778730 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.778693    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.837685 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.837642    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 05:41:18.838221 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.837821    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 05:41:18.879084 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.879055    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.903324 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.903216    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 05:36:18 +0000 UTC" deadline="2028-03-03 00:14:09.786869746 +0000 UTC"
Jul 17 05:41:18.903324 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.903271    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14274h32m50.883604036s"
Jul 17 05:41:18.979237 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:18.979196    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:18.992262 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:18.992234    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 05:41:19.008063 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.008037    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 05:41:19.074161 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:19.074124    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb714b8177e3d9d42f58f7241552dc9e9.slice/crio-db27a5e6f12e67fbe0799e5e7eca8c9ca6131d90c6e729fc12efd25a7a738159 WatchSource:0}: Error finding container db27a5e6f12e67fbe0799e5e7eca8c9ca6131d90c6e729fc12efd25a7a738159: Status 404 returned error can't find the container with id db27a5e6f12e67fbe0799e5e7eca8c9ca6131d90c6e729fc12efd25a7a738159
Jul 17 05:41:19.074680 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:19.074658    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa1ec63b2d121d5f010fce817a84937.slice/crio-3acea623a6cf1d5f5a26ff5c0a3144b22dbffeacbfbaf734fed79049d72c5648 WatchSource:0}: Error finding container 3acea623a6cf1d5f5a26ff5c0a3144b22dbffeacbfbaf734fed79049d72c5648: Status 404 returned error can't find the container with id 3acea623a6cf1d5f5a26ff5c0a3144b22dbffeacbfbaf734fed79049d72c5648
Jul 17 05:41:19.079355 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:19.079333    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:19.080032 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.080013    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:41:19.185548 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:19.185439    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:19.285541 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:19.285509    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-96.ec2.internal\" not found"
Jul 17 05:41:19.373912 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.373887    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 05:41:19.396295 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.396273    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:19.407833 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.407812    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 05:41:19.409240 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.409227    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-96.ec2.internal"
Jul 17 05:41:19.420123 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.420105    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 05:41:19.875124 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.875094    2558 apiserver.go:52] "Watching apiserver"
Jul 17 05:41:19.890031 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.889999    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 05:41:19.892915 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.892884    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-jtnfh","openshift-multus/network-metrics-daemon-tnctb","openshift-ovn-kubernetes/ovnkube-node-fj8r8","kube-system/konnectivity-agent-crk4m","kube-system/kube-apiserver-proxy-ip-10-0-140-96.ec2.internal","openshift-cluster-node-tuning-operator/tuned-72jjp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal","openshift-network-diagnostics/network-check-target-85qp6","openshift-network-operator/iptables-alerter-vzmnt","kube-system/global-pull-secret-syncer-h2lgh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl","openshift-dns/node-resolver-lfzc2","openshift-image-registry/node-ca-6drsh","openshift-multus/multus-additional-cni-plugins-rflw8"]
Jul 17 05:41:19.894608 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.894585    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:19.894709 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:19.894666    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:19.897054 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.896600    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:19.897162 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.897074    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 05:41:19.900261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.898266    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.900261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.899956    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 05:41:19.900261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.900027    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 05:41:19.900261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.900186    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 05:41:19.900510 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.900334    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-52s9f\""
Jul 17 05:41:19.901074 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.901005    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gfxfr\""
Jul 17 05:41:19.901074 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.901059    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 05:41:19.901378 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.901358    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 05:41:19.901445 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.901411    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:19.901590 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.901573    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 05:41:19.901894 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.901879    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 05:41:19.902935 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.902913    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 05:41:19.903158 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.903140    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 05:41:19.904000 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.903979    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:19.906367 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.905882    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 05:41:19.906367 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.906139    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lxcrg\""
Jul 17 05:41:19.906367 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.906333    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 05:41:19.906914 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.906806    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qjr4m\""
Jul 17 05:41:19.907034 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.907018    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 05:41:19.907169 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.907155    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 05:41:19.907433 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.907417    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:19.907557 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.907538    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 05:41:19.907634 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.907599    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 05:41:19.907689 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.907549    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.908690    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1b659b41-1e15-4c23-95b6-0396eadae285-dbus\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.908923    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-registration-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909156    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909185    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-device-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909212    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/56e41793-4279-46d8-84a0-ad435e812001-ovnkube-script-lib\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909237    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2lsx\" (UniqueName: \"kubernetes.io/projected/44bef013-3171-42a3-8771-13dda23de637-kube-api-access-n2lsx\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909263    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7tzc\" (UniqueName: \"kubernetes.io/projected/56e41793-4279-46d8-84a0-ad435e812001-kube-api-access-x7tzc\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909289    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909314    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-kubelet\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909346    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1b659b41-1e15-4c23-95b6-0396eadae285-kubelet-config\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909371    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-systemd-units\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909393    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-run-netns\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909416    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-var-lib-openvswitch\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909433    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-etc-openvswitch\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909452    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-run-ovn-kubernetes\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.909605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909470    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-slash\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909484    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-cni-bin\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909498    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-cni-netd\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909549    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-etc-selinux\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909587    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-run-systemd\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909612    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-node-log\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909641    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-sys-fs\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909663    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-run-ovn\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909687    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-log-socket\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909710    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909765    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-socket-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909905    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-run-openvswitch\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909961    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/56e41793-4279-46d8-84a0-ad435e812001-ovnkube-config\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909983    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vzmnt"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.909989    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/56e41793-4279-46d8-84a0-ad435e812001-env-overrides\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.910015    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.910023    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/56e41793-4279-46d8-84a0-ad435e812001-ovn-node-metrics-cert\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.910300    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ht6v2\""
Jul 17 05:41:19.910403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.910330    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 05:41:19.912959 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.912614    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 05:41:19.913097 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.913081    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hl865\""
Jul 17 05:41:19.913323 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.913306    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 05:41:19.913506 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.913490    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 05:41:19.914238 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.914221    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6drsh"
Jul 17 05:41:19.914646 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.914626    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 05:41:19.916278 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.914976    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:19.916278 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:19.915102    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:19.924023 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.923990    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 05:41:19.924122 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.924083    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 05:41:19.924362 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.923991    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 05:41:19.924471 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.924393    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mf65m\""
Jul 17 05:41:19.924899 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.924877    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:19.927129 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.927109    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:19.927219 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:19.927173    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:19.927769 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.927660    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 05:41:19.927769 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.927705    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-66c5t\""
Jul 17 05:41:19.928419 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.928401    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lfzc2"
Jul 17 05:41:19.931185 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.931156    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 05:41:19.931281 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.931241    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wwp7t\""
Jul 17 05:41:19.931281 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.931256    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 05:41:19.935540 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.935517    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pghm5"
Jul 17 05:41:19.941969 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.941946    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pghm5"
Jul 17 05:41:19.998183 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:19.998139    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010171    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-run-ovn\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010211    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq2z4\" (UniqueName: \"kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4\") pod \"network-check-target-85qp6\" (UID: \"b24412fb-094a-4e0f-8de0-b848ebaf4323\") " pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010237    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a31ea27-5883-4df9-9032-72ad9c0ee4ea-hosts-file\") pod \"node-resolver-lfzc2\" (UID: \"2a31ea27-5883-4df9-9032-72ad9c0ee4ea\") " pod="openshift-dns/node-resolver-lfzc2"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010340    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a31ea27-5883-4df9-9032-72ad9c0ee4ea-tmp-dir\") pod \"node-resolver-lfzc2\" (UID: \"2a31ea27-5883-4df9-9032-72ad9c0ee4ea\") " pod="openshift-dns/node-resolver-lfzc2"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010367    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-cnibin\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010324    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-run-ovn\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010391    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010411    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kcwx\" (UniqueName: \"kubernetes.io/projected/e92b27e5-e21d-46ed-8a43-212e4f441cae-kube-api-access-5kcwx\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010430    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-run-netns\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010457    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010508    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/56e41793-4279-46d8-84a0-ad435e812001-ovn-node-metrics-cert\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010537    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-system-cni-dir\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010543    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010575    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-multus-cni-dir\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010599    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5626b442-c1f4-4444-92d8-fb67270cab22-cni-binary-copy\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010621    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-var-lib-cni-bin\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.011239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010643    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t597\" (UniqueName: \"kubernetes.io/projected/5626b442-c1f4-4444-92d8-fb67270cab22-kube-api-access-2t597\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010669    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-registration-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010695    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-sysconfig\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-var-lib-kubelet\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010739    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-hostroot\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010765    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/56e41793-4279-46d8-84a0-ad435e812001-ovnkube-script-lib\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010789    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2fbb5583-456e-4c0a-b9e8-af3dd2c40797-host-slash\") pod \"iptables-alerter-vzmnt\" (UID: \"2fbb5583-456e-4c0a-b9e8-af3dd2c40797\") " pod="openshift-network-operator/iptables-alerter-vzmnt"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvv7c\" (UniqueName: \"kubernetes.io/projected/19e2915f-ebfc-45ff-9452-9e0c35a105ca-kube-api-access-cvv7c\") pod \"node-ca-6drsh\" (UID: \"19e2915f-ebfc-45ff-9452-9e0c35a105ca\") " pod="openshift-image-registry/node-ca-6drsh"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010851    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c14d858-96a0-4454-9f78-f2a403da7d11-cni-binary-copy\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010874    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-host\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010889    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010898    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-run-k8s-cni-cncf-io\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010931    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5626b442-c1f4-4444-92d8-fb67270cab22-multus-daemon-config\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010954    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7tzc\" (UniqueName: \"kubernetes.io/projected/56e41793-4279-46d8-84a0-ad435e812001-kube-api-access-x7tzc\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.010989    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sngn4\" (UniqueName: \"kubernetes.io/projected/2a31ea27-5883-4df9-9032-72ad9c0ee4ea-kube-api-access-sngn4\") pod \"node-resolver-lfzc2\" (UID: \"2a31ea27-5883-4df9-9032-72ad9c0ee4ea\") " pod="openshift-dns/node-resolver-lfzc2"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011012    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2fbb5583-456e-4c0a-b9e8-af3dd2c40797-iptables-alerter-script\") pod \"iptables-alerter-vzmnt\" (UID: \"2fbb5583-456e-4c0a-b9e8-af3dd2c40797\") " pod="openshift-network-operator/iptables-alerter-vzmnt"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011041    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-os-release\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.012132 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011062    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-cnibin\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011083    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-os-release\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011106    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011128    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-kubelet\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011165    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1b659b41-1e15-4c23-95b6-0396eadae285-kubelet-config\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-systemd-units\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011237    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-run-ovn-kubernetes\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011259    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-slash\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-lib-modules\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-etc-selinux\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011348    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-run-systemd\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011370    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-node-log\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011396    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c14d858-96a0-4454-9f78-f2a403da7d11-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011424    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-sysctl-d\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011462    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-run-multus-certs\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011497    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-sys-fs\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011521    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-log-socket\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.013005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011567    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011594    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-modprobe-d\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011621    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-systemd\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011644    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-tmp\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011671    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-socket-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011698    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-run-openvswitch\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011724    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/56e41793-4279-46d8-84a0-ad435e812001-ovnkube-config\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011750    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/56e41793-4279-46d8-84a0-ad435e812001-env-overrides\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011775    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a773d59-6bb2-4da5-97e6-53ae51a55cd7-konnectivity-ca\") pod \"konnectivity-agent-crk4m\" (UID: \"5a773d59-6bb2-4da5-97e6-53ae51a55cd7\") " pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011802    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnhw7\" (UniqueName: \"kubernetes.io/projected/9c14d858-96a0-4454-9f78-f2a403da7d11-kube-api-access-pnhw7\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-tuned\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011854    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q56xj\" (UniqueName: \"kubernetes.io/projected/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-kube-api-access-q56xj\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1b659b41-1e15-4c23-95b6-0396eadae285-dbus\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011908    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lngkg\" (UniqueName: \"kubernetes.io/projected/2fbb5583-456e-4c0a-b9e8-af3dd2c40797-kube-api-access-lngkg\") pod \"iptables-alerter-vzmnt\" (UID: \"2fbb5583-456e-4c0a-b9e8-af3dd2c40797\") " pod="openshift-network-operator/iptables-alerter-vzmnt"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c14d858-96a0-4454-9f78-f2a403da7d11-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.011962    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-kubernetes\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.013843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012027    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-systemd-units\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012041    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-run-ovn-kubernetes\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012088    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-slash\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012138    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-registration-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012200    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-etc-selinux\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-run-systemd\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012290    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-node-log\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012368    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-sys-fs\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-log-socket\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012646    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-socket-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012691    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/56e41793-4279-46d8-84a0-ad435e812001-ovnkube-script-lib\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.012698    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-run-openvswitch\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013116    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/56e41793-4279-46d8-84a0-ad435e812001-ovnkube-config\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013120    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/56e41793-4279-46d8-84a0-ad435e812001-env-overrides\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013301    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1b659b41-1e15-4c23-95b6-0396eadae285-dbus\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013394    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-multus-socket-dir-parent\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013415    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1b659b41-1e15-4c23-95b6-0396eadae285-kubelet-config\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:20.014571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013446    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013455    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013471    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-kubelet\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-device-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013548    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-var-lib-kubelet\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013573    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44bef013-3171-42a3-8771-13dda23de637-device-dir\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013579    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2lsx\" (UniqueName: \"kubernetes.io/projected/44bef013-3171-42a3-8771-13dda23de637-kube-api-access-n2lsx\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.013774    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013799    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a773d59-6bb2-4da5-97e6-53ae51a55cd7-agent-certs\") pod \"konnectivity-agent-crk4m\" (UID: \"5a773d59-6bb2-4da5-97e6-53ae51a55cd7\") " pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.013861    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret podName:1b659b41-1e15-4c23-95b6-0396eadae285 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:20.513828726 +0000 UTC m=+2.955767285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret") pod "global-pull-secret-syncer-h2lgh" (UID: "1b659b41-1e15-4c23-95b6-0396eadae285") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013882    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19e2915f-ebfc-45ff-9452-9e0c35a105ca-host\") pod \"node-ca-6drsh\" (UID: \"19e2915f-ebfc-45ff-9452-9e0c35a105ca\") " pod="openshift-image-registry/node-ca-6drsh"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013911    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19e2915f-ebfc-45ff-9452-9e0c35a105ca-serviceca\") pod \"node-ca-6drsh\" (UID: \"19e2915f-ebfc-45ff-9452-9e0c35a105ca\") " pod="openshift-image-registry/node-ca-6drsh"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-sysctl-conf\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013957    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-run\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.013985    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-run-netns\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014011    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-var-lib-openvswitch\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014061    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-etc-openvswitch\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.015310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014072    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-run-netns\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.016172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014084    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-var-lib-cni-multus\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.016172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-etc-openvswitch\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.016172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014120    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-var-lib-openvswitch\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.016172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014152    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-etc-kubernetes\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.016172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014182    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-cni-bin\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.016172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014212    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-cni-netd\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.016172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014235    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-system-cni-dir\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.016172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014259    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-multus-conf-dir\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.016172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014278    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-cni-netd\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.016172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014282    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-sys\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.016172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.014360    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56e41793-4279-46d8-84a0-ad435e812001-host-cni-bin\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.017097 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.017076    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/56e41793-4279-46d8-84a0-ad435e812001-ovn-node-metrics-cert\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.027636 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.023391    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7tzc\" (UniqueName: \"kubernetes.io/projected/56e41793-4279-46d8-84a0-ad435e812001-kube-api-access-x7tzc\") pod \"ovnkube-node-fj8r8\" (UID: \"56e41793-4279-46d8-84a0-ad435e812001\") " pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.028233 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.028199    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2lsx\" (UniqueName: \"kubernetes.io/projected/44bef013-3171-42a3-8771-13dda23de637-kube-api-access-n2lsx\") pod \"aws-ebs-csi-driver-node-hhxcl\" (UID: \"44bef013-3171-42a3-8771-13dda23de637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.077426 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.077380    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal" event={"ID":"5aa1ec63b2d121d5f010fce817a84937","Type":"ContainerStarted","Data":"3acea623a6cf1d5f5a26ff5c0a3144b22dbffeacbfbaf734fed79049d72c5648"}
Jul 17 05:41:20.079226 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.079200    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-96.ec2.internal" event={"ID":"b714b8177e3d9d42f58f7241552dc9e9","Type":"ContainerStarted","Data":"db27a5e6f12e67fbe0799e5e7eca8c9ca6131d90c6e729fc12efd25a7a738159"}
Jul 17 05:41:20.114959 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.114925    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sngn4\" (UniqueName: \"kubernetes.io/projected/2a31ea27-5883-4df9-9032-72ad9c0ee4ea-kube-api-access-sngn4\") pod \"node-resolver-lfzc2\" (UID: \"2a31ea27-5883-4df9-9032-72ad9c0ee4ea\") " pod="openshift-dns/node-resolver-lfzc2"
Jul 17 05:41:20.115102 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.114963    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2fbb5583-456e-4c0a-b9e8-af3dd2c40797-iptables-alerter-script\") pod \"iptables-alerter-vzmnt\" (UID: \"2fbb5583-456e-4c0a-b9e8-af3dd2c40797\") " pod="openshift-network-operator/iptables-alerter-vzmnt"
Jul 17 05:41:20.115102 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.114987    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-os-release\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.115102 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115012    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-cnibin\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.115102 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115035    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-os-release\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.115102 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115067    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-lib-modules\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.115102 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115094    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c14d858-96a0-4454-9f78-f2a403da7d11-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115117    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-sysctl-d\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115140    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-run-multus-certs\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115182    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115206    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-modprobe-d\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-systemd\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-tmp\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115279    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a773d59-6bb2-4da5-97e6-53ae51a55cd7-konnectivity-ca\") pod \"konnectivity-agent-crk4m\" (UID: \"5a773d59-6bb2-4da5-97e6-53ae51a55cd7\") " pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115304    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnhw7\" (UniqueName: \"kubernetes.io/projected/9c14d858-96a0-4454-9f78-f2a403da7d11-kube-api-access-pnhw7\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115329    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-tuned\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115354    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q56xj\" (UniqueName: \"kubernetes.io/projected/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-kube-api-access-q56xj\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115378    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lngkg\" (UniqueName: \"kubernetes.io/projected/2fbb5583-456e-4c0a-b9e8-af3dd2c40797-kube-api-access-lngkg\") pod \"iptables-alerter-vzmnt\" (UID: \"2fbb5583-456e-4c0a-b9e8-af3dd2c40797\") " pod="openshift-network-operator/iptables-alerter-vzmnt"
Jul 17 05:41:20.115414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115404    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c14d858-96a0-4454-9f78-f2a403da7d11-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115434    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-kubernetes\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-multus-socket-dir-parent\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115526    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-var-lib-kubelet\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115553    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a773d59-6bb2-4da5-97e6-53ae51a55cd7-agent-certs\") pod \"konnectivity-agent-crk4m\" (UID: \"5a773d59-6bb2-4da5-97e6-53ae51a55cd7\") " pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115578    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19e2915f-ebfc-45ff-9452-9e0c35a105ca-host\") pod \"node-ca-6drsh\" (UID: \"19e2915f-ebfc-45ff-9452-9e0c35a105ca\") " pod="openshift-image-registry/node-ca-6drsh"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115601    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19e2915f-ebfc-45ff-9452-9e0c35a105ca-serviceca\") pod \"node-ca-6drsh\" (UID: \"19e2915f-ebfc-45ff-9452-9e0c35a105ca\") " pod="openshift-image-registry/node-ca-6drsh"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-sysctl-conf\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-run\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-var-lib-cni-multus\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115713    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-etc-kubernetes\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115740    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-system-cni-dir\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115766    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-multus-conf-dir\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115790    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-sys\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115816    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq2z4\" (UniqueName: \"kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4\") pod \"network-check-target-85qp6\" (UID: \"b24412fb-094a-4e0f-8de0-b848ebaf4323\") " pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115842    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a31ea27-5883-4df9-9032-72ad9c0ee4ea-hosts-file\") pod \"node-resolver-lfzc2\" (UID: \"2a31ea27-5883-4df9-9032-72ad9c0ee4ea\") " pod="openshift-dns/node-resolver-lfzc2"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115864    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a31ea27-5883-4df9-9032-72ad9c0ee4ea-tmp-dir\") pod \"node-resolver-lfzc2\" (UID: \"2a31ea27-5883-4df9-9032-72ad9c0ee4ea\") " pod="openshift-dns/node-resolver-lfzc2"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115890    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-cnibin\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.116051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115919    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115947    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kcwx\" (UniqueName: \"kubernetes.io/projected/e92b27e5-e21d-46ed-8a43-212e4f441cae-kube-api-access-5kcwx\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.115975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-run-netns\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116003    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-system-cni-dir\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116028    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-multus-cni-dir\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116055    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5626b442-c1f4-4444-92d8-fb67270cab22-cni-binary-copy\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116080    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-var-lib-cni-bin\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116106    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t597\" (UniqueName: \"kubernetes.io/projected/5626b442-c1f4-4444-92d8-fb67270cab22-kube-api-access-2t597\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116131    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-sysconfig\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116156    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-var-lib-kubelet\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116181    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-hostroot\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116208    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2fbb5583-456e-4c0a-b9e8-af3dd2c40797-host-slash\") pod \"iptables-alerter-vzmnt\" (UID: \"2fbb5583-456e-4c0a-b9e8-af3dd2c40797\") " pod="openshift-network-operator/iptables-alerter-vzmnt"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116234    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvv7c\" (UniqueName: \"kubernetes.io/projected/19e2915f-ebfc-45ff-9452-9e0c35a105ca-kube-api-access-cvv7c\") pod \"node-ca-6drsh\" (UID: \"19e2915f-ebfc-45ff-9452-9e0c35a105ca\") " pod="openshift-image-registry/node-ca-6drsh"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116261    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c14d858-96a0-4454-9f78-f2a403da7d11-cni-binary-copy\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116287    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-host\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116312    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-run-k8s-cni-cncf-io\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.116338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5626b442-c1f4-4444-92d8-fb67270cab22-multus-daemon-config\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.117019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.117011    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5626b442-c1f4-4444-92d8-fb67270cab22-multus-daemon-config\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.117942 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.117776    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2fbb5583-456e-4c0a-b9e8-af3dd2c40797-iptables-alerter-script\") pod \"iptables-alerter-vzmnt\" (UID: \"2fbb5583-456e-4c0a-b9e8-af3dd2c40797\") " pod="openshift-network-operator/iptables-alerter-vzmnt"
Jul 17 05:41:20.117942 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.117862    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-os-release\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.117942 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.117915    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-cnibin\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.118088 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.117965    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-os-release\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.118088 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.118066    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-lib-modules\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.118484 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.118463    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c14d858-96a0-4454-9f78-f2a403da7d11-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.118660 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.118612    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-sysctl-d\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.118763 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.118664    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-run-multus-certs\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.118826 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.118797    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.118927 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.118889    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-modprobe-d\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.118985 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.118943    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-systemd\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.120715 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.120690    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a773d59-6bb2-4da5-97e6-53ae51a55cd7-konnectivity-ca\") pod \"konnectivity-agent-crk4m\" (UID: \"5a773d59-6bb2-4da5-97e6-53ae51a55cd7\") " pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:20.121283 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.121258    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19e2915f-ebfc-45ff-9452-9e0c35a105ca-host\") pod \"node-ca-6drsh\" (UID: \"19e2915f-ebfc-45ff-9452-9e0c35a105ca\") " pod="openshift-image-registry/node-ca-6drsh"
Jul 17 05:41:20.122147 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.122120    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c14d858-96a0-4454-9f78-f2a403da7d11-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.122245 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.122201    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-kubernetes\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.122333 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.122255    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-multus-socket-dir-parent\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.122333 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.122303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-var-lib-kubelet\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.123602 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.122878    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a31ea27-5883-4df9-9032-72ad9c0ee4ea-hosts-file\") pod \"node-resolver-lfzc2\" (UID: \"2a31ea27-5883-4df9-9032-72ad9c0ee4ea\") " pod="openshift-dns/node-resolver-lfzc2"
Jul 17 05:41:20.123602 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123083    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-var-lib-cni-multus\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.123602 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123161    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a31ea27-5883-4df9-9032-72ad9c0ee4ea-tmp-dir\") pod \"node-resolver-lfzc2\" (UID: \"2a31ea27-5883-4df9-9032-72ad9c0ee4ea\") " pod="openshift-dns/node-resolver-lfzc2"
Jul 17 05:41:20.123602 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123242    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-sysconfig\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.123602 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123264    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-cnibin\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.123602 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123313    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-hostroot\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.123602 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.123347    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:20.123602 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123363    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2fbb5583-456e-4c0a-b9e8-af3dd2c40797-host-slash\") pod \"iptables-alerter-vzmnt\" (UID: \"2fbb5583-456e-4c0a-b9e8-af3dd2c40797\") " pod="openshift-network-operator/iptables-alerter-vzmnt"
Jul 17 05:41:20.123602 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.123407    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs podName:e92b27e5-e21d-46ed-8a43-212e4f441cae nodeName:}" failed. No retries permitted until 2026-07-17 05:41:20.623389878 +0000 UTC m=+3.065328452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs") pod "network-metrics-daemon-tnctb" (UID: "e92b27e5-e21d-46ed-8a43-212e4f441cae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:20.124160 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123630    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19e2915f-ebfc-45ff-9452-9e0c35a105ca-serviceca\") pod \"node-ca-6drsh\" (UID: \"19e2915f-ebfc-45ff-9452-9e0c35a105ca\") " pod="openshift-image-registry/node-ca-6drsh"
Jul 17 05:41:20.124160 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123640    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-run-netns\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.124160 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123685    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c14d858-96a0-4454-9f78-f2a403da7d11-system-cni-dir\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.124160 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123746    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-multus-cni-dir\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.124160 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123781    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-sysctl-conf\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.124160 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.123835    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-run\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.124160 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.124002    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-tmp\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.124160 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.124061    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-system-cni-dir\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.124160 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.124102    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-etc-kubernetes\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.124160 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.124140    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-var-lib-kubelet\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.124740 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.124184    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-var-lib-cni-bin\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.124740 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.124229    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5626b442-c1f4-4444-92d8-fb67270cab22-cni-binary-copy\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.124740 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.124238    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-multus-conf-dir\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.124740 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.124299    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-sys\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.124740 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.124300    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-host\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.124740 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.124364    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5626b442-c1f4-4444-92d8-fb67270cab22-host-run-k8s-cni-cncf-io\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.124740 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.124718    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c14d858-96a0-4454-9f78-f2a403da7d11-cni-binary-copy\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.125462 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.125380    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a773d59-6bb2-4da5-97e6-53ae51a55cd7-agent-certs\") pod \"konnectivity-agent-crk4m\" (UID: \"5a773d59-6bb2-4da5-97e6-53ae51a55cd7\") " pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:20.126231 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.126209    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:41:20.126340 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.126240    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:41:20.126340 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.126254    2558 projected.go:196] Error preparing data for projected volume kube-api-access-sq2z4 for pod openshift-network-diagnostics/network-check-target-85qp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:20.126537 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.126345    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4 podName:b24412fb-094a-4e0f-8de0-b848ebaf4323 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:20.62632739 +0000 UTC m=+3.068265958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sq2z4" (UniqueName: "kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4") pod "network-check-target-85qp6" (UID: "b24412fb-094a-4e0f-8de0-b848ebaf4323") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:20.128665 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.128428    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-etc-tuned\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.130966 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.130771    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sngn4\" (UniqueName: \"kubernetes.io/projected/2a31ea27-5883-4df9-9032-72ad9c0ee4ea-kube-api-access-sngn4\") pod \"node-resolver-lfzc2\" (UID: \"2a31ea27-5883-4df9-9032-72ad9c0ee4ea\") " pod="openshift-dns/node-resolver-lfzc2"
Jul 17 05:41:20.131062 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.130995    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q56xj\" (UniqueName: \"kubernetes.io/projected/90b3c468-7bbf-41b8-9a4a-7922b74b2d47-kube-api-access-q56xj\") pod \"tuned-72jjp\" (UID: \"90b3c468-7bbf-41b8-9a4a-7922b74b2d47\") " pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.133589 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.133427    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnhw7\" (UniqueName: \"kubernetes.io/projected/9c14d858-96a0-4454-9f78-f2a403da7d11-kube-api-access-pnhw7\") pod \"multus-additional-cni-plugins-rflw8\" (UID: \"9c14d858-96a0-4454-9f78-f2a403da7d11\") " pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.134297 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.134273    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lngkg\" (UniqueName: \"kubernetes.io/projected/2fbb5583-456e-4c0a-b9e8-af3dd2c40797-kube-api-access-lngkg\") pod \"iptables-alerter-vzmnt\" (UID: \"2fbb5583-456e-4c0a-b9e8-af3dd2c40797\") " pod="openshift-network-operator/iptables-alerter-vzmnt"
Jul 17 05:41:20.136863 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.136821    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvv7c\" (UniqueName: \"kubernetes.io/projected/19e2915f-ebfc-45ff-9452-9e0c35a105ca-kube-api-access-cvv7c\") pod \"node-ca-6drsh\" (UID: \"19e2915f-ebfc-45ff-9452-9e0c35a105ca\") " pod="openshift-image-registry/node-ca-6drsh"
Jul 17 05:41:20.140807 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.140787    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kcwx\" (UniqueName: \"kubernetes.io/projected/e92b27e5-e21d-46ed-8a43-212e4f441cae-kube-api-access-5kcwx\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:20.141426 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.141382    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t597\" (UniqueName: \"kubernetes.io/projected/5626b442-c1f4-4444-92d8-fb67270cab22-kube-api-access-2t597\") pod \"multus-jtnfh\" (UID: \"5626b442-c1f4-4444-92d8-fb67270cab22\") " pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.175709 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.175687    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 05:41:20.214347 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.214320    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl"
Jul 17 05:41:20.233533 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.233507    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:20.241485 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.241461    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:20.248445 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.248425    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rflw8"
Jul 17 05:41:20.265446 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.265292    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-72jjp"
Jul 17 05:41:20.273581 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.273554    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vzmnt"
Jul 17 05:41:20.283629 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.283605    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6drsh"
Jul 17 05:41:20.293680 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.293659    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jtnfh"
Jul 17 05:41:20.301701 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.301679    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lfzc2"
Jul 17 05:41:20.519858 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.519817    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:20.519994 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.519949    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:20.520039 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.520011    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret podName:1b659b41-1e15-4c23-95b6-0396eadae285 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:21.519996229 +0000 UTC m=+3.961934786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret") pod "global-pull-secret-syncer-h2lgh" (UID: "1b659b41-1e15-4c23-95b6-0396eadae285") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:20.721067 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.721002    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq2z4\" (UniqueName: \"kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4\") pod \"network-check-target-85qp6\" (UID: \"b24412fb-094a-4e0f-8de0-b848ebaf4323\") " pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:20.721067 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.721034    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:20.721229 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.721153    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:20.721229 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.721171    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:41:20.721229 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.721208    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:41:20.721229 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.721222    2558 projected.go:196] Error preparing data for projected volume kube-api-access-sq2z4 for pod openshift-network-diagnostics/network-check-target-85qp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:20.721439 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.721204    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs podName:e92b27e5-e21d-46ed-8a43-212e4f441cae nodeName:}" failed. No retries permitted until 2026-07-17 05:41:21.721191048 +0000 UTC m=+4.163129601 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs") pod "network-metrics-daemon-tnctb" (UID: "e92b27e5-e21d-46ed-8a43-212e4f441cae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:20.721439 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:20.721278    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4 podName:b24412fb-094a-4e0f-8de0-b848ebaf4323 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:21.721265659 +0000 UTC m=+4.163204213 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sq2z4" (UniqueName: "kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4") pod "network-check-target-85qp6" (UID: "b24412fb-094a-4e0f-8de0-b848ebaf4323") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:20.943885 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.943819    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 05:36:20 +0000 UTC" deadline="2028-04-22 21:49:28.711043544 +0000 UTC"
Jul 17 05:41:20.943885 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:20.943886    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15496h8m7.767168059s"
Jul 17 05:41:21.027771 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:21.027734    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b3c468_7bbf_41b8_9a4a_7922b74b2d47.slice/crio-20dfa9d83f07f899282ec81c3a35c53902431a7b2575dd88225a7a7139d97de1 WatchSource:0}: Error finding container 20dfa9d83f07f899282ec81c3a35c53902431a7b2575dd88225a7a7139d97de1: Status 404 returned error can't find the container with id 20dfa9d83f07f899282ec81c3a35c53902431a7b2575dd88225a7a7139d97de1
Jul 17 05:41:21.031703 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:21.031680    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c14d858_96a0_4454_9f78_f2a403da7d11.slice/crio-98aa7fdfc4ecb37fe05531eccb2e6ba1c7db1705fe805ff5c899b83e207b4dfe WatchSource:0}: Error finding container 98aa7fdfc4ecb37fe05531eccb2e6ba1c7db1705fe805ff5c899b83e207b4dfe: Status 404 returned error can't find the container with id 98aa7fdfc4ecb37fe05531eccb2e6ba1c7db1705fe805ff5c899b83e207b4dfe
Jul 17 05:41:21.033397 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:21.033366    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a31ea27_5883_4df9_9032_72ad9c0ee4ea.slice/crio-dd03d49d4d1778695511275e9a95940f1ef40a3d00edce9283815a8fc62adf78 WatchSource:0}: Error finding container dd03d49d4d1778695511275e9a95940f1ef40a3d00edce9283815a8fc62adf78: Status 404 returned error can't find the container with id dd03d49d4d1778695511275e9a95940f1ef40a3d00edce9283815a8fc62adf78
Jul 17 05:41:21.034591 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:21.034567    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a773d59_6bb2_4da5_97e6_53ae51a55cd7.slice/crio-b95ec8a7f53b50e71ad82a907d821537f60586cdf195dfb39dd4f9ea4271b463 WatchSource:0}: Error finding container b95ec8a7f53b50e71ad82a907d821537f60586cdf195dfb39dd4f9ea4271b463: Status 404 returned error can't find the container with id b95ec8a7f53b50e71ad82a907d821537f60586cdf195dfb39dd4f9ea4271b463
Jul 17 05:41:21.035614 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:21.035448    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5626b442_c1f4_4444_92d8_fb67270cab22.slice/crio-719a794e57f2df642939a6636a3e6bdfbcb6896ab82b3f67e019c1fd3734115a WatchSource:0}: Error finding container 719a794e57f2df642939a6636a3e6bdfbcb6896ab82b3f67e019c1fd3734115a: Status 404 returned error can't find the container with id 719a794e57f2df642939a6636a3e6bdfbcb6896ab82b3f67e019c1fd3734115a
Jul 17 05:41:21.037091 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:21.037068    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56e41793_4279_46d8_84a0_ad435e812001.slice/crio-376166fb07d47700568cf7ce6b4793251bc469043124e21180b66c4135457318 WatchSource:0}: Error finding container 376166fb07d47700568cf7ce6b4793251bc469043124e21180b66c4135457318: Status 404 returned error can't find the container with id 376166fb07d47700568cf7ce6b4793251bc469043124e21180b66c4135457318
Jul 17 05:41:21.072609 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.072445    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:21.072737 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:21.072716    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:21.083590 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.083563    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-crk4m" event={"ID":"5a773d59-6bb2-4da5-97e6-53ae51a55cd7","Type":"ContainerStarted","Data":"b95ec8a7f53b50e71ad82a907d821537f60586cdf195dfb39dd4f9ea4271b463"}
Jul 17 05:41:21.084767 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.084742    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rflw8" event={"ID":"9c14d858-96a0-4454-9f78-f2a403da7d11","Type":"ContainerStarted","Data":"98aa7fdfc4ecb37fe05531eccb2e6ba1c7db1705fe805ff5c899b83e207b4dfe"}
Jul 17 05:41:21.086121 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.086093    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-72jjp" event={"ID":"90b3c468-7bbf-41b8-9a4a-7922b74b2d47","Type":"ContainerStarted","Data":"20dfa9d83f07f899282ec81c3a35c53902431a7b2575dd88225a7a7139d97de1"}
Jul 17 05:41:21.087387 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.087365    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jtnfh" event={"ID":"5626b442-c1f4-4444-92d8-fb67270cab22","Type":"ContainerStarted","Data":"719a794e57f2df642939a6636a3e6bdfbcb6896ab82b3f67e019c1fd3734115a"}
Jul 17 05:41:21.088616 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.088586    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl" event={"ID":"44bef013-3171-42a3-8771-13dda23de637","Type":"ContainerStarted","Data":"1b985c71b58724f4b9d434ddf0cb4fc2bbc47f6dafa2b2642215e8571bf348cb"}
Jul 17 05:41:21.089665 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.089638    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6drsh" event={"ID":"19e2915f-ebfc-45ff-9452-9e0c35a105ca","Type":"ContainerStarted","Data":"9febd9bec1533503b94d992d203f583557785d07c0f9b75a8f151b77b24c0d27"}
Jul 17 05:41:21.090984 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.090952    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8" event={"ID":"56e41793-4279-46d8-84a0-ad435e812001","Type":"ContainerStarted","Data":"376166fb07d47700568cf7ce6b4793251bc469043124e21180b66c4135457318"}
Jul 17 05:41:21.092171 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.092151    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vzmnt" event={"ID":"2fbb5583-456e-4c0a-b9e8-af3dd2c40797","Type":"ContainerStarted","Data":"2887ccc74baae9220070915b325a2560d5016c8396579e67fce44c9f2fd0ba5a"}
Jul 17 05:41:21.093234 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.093214    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lfzc2" event={"ID":"2a31ea27-5883-4df9-9032-72ad9c0ee4ea","Type":"ContainerStarted","Data":"dd03d49d4d1778695511275e9a95940f1ef40a3d00edce9283815a8fc62adf78"}
Jul 17 05:41:21.527270 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.527192    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:21.527431 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:21.527387    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:21.527479 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:21.527455    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret podName:1b659b41-1e15-4c23-95b6-0396eadae285 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:23.527435005 +0000 UTC m=+5.969373616 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret") pod "global-pull-secret-syncer-h2lgh" (UID: "1b659b41-1e15-4c23-95b6-0396eadae285") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:21.728427 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.728387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq2z4\" (UniqueName: \"kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4\") pod \"network-check-target-85qp6\" (UID: \"b24412fb-094a-4e0f-8de0-b848ebaf4323\") " pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:21.728615 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.728436    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:21.728678 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:21.728625    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:21.728748 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:21.728690    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs podName:e92b27e5-e21d-46ed-8a43-212e4f441cae nodeName:}" failed. No retries permitted until 2026-07-17 05:41:23.728669356 +0000 UTC m=+6.170607913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs") pod "network-metrics-daemon-tnctb" (UID: "e92b27e5-e21d-46ed-8a43-212e4f441cae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:21.729170 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:21.729120    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:41:21.729170 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:21.729142    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:41:21.729170 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:21.729156    2558 projected.go:196] Error preparing data for projected volume kube-api-access-sq2z4 for pod openshift-network-diagnostics/network-check-target-85qp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:21.729369 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:21.729202    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4 podName:b24412fb-094a-4e0f-8de0-b848ebaf4323 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:23.729186064 +0000 UTC m=+6.171124623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sq2z4" (UniqueName: "kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4") pod "network-check-target-85qp6" (UID: "b24412fb-094a-4e0f-8de0-b848ebaf4323") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:21.944171 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.944059    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 05:36:20 +0000 UTC" deadline="2027-12-28 11:59:27.810973703 +0000 UTC"
Jul 17 05:41:21.944171 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:21.944121    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12702h18m5.86685743s"
Jul 17 05:41:22.076045 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:22.075333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:22.076045 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:22.075460    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:22.076045 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:22.075893    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:22.076045 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:22.075998    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:22.119443 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:22.119409    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-96.ec2.internal" event={"ID":"b714b8177e3d9d42f58f7241552dc9e9","Type":"ContainerStarted","Data":"7cbdd80047ad77e5f423550251755a7c15edeaeb00f0d174517d49006280a419"}
Jul 17 05:41:22.128466 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:22.128253    2558 generic.go:356] "Generic (PLEG): container finished" podID="5aa1ec63b2d121d5f010fce817a84937" containerID="696e30653cf28bc158a07369eb1f009389882012386fd5cf6c953ae73007e7f4" exitCode=0
Jul 17 05:41:22.128466 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:22.128288    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal" event={"ID":"5aa1ec63b2d121d5f010fce817a84937","Type":"ContainerDied","Data":"696e30653cf28bc158a07369eb1f009389882012386fd5cf6c953ae73007e7f4"}
Jul 17 05:41:22.135376 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:22.135319    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-96.ec2.internal" podStartSLOduration=2.135301664 podStartE2EDuration="2.135301664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:41:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:22.134489736 +0000 UTC m=+4.576428327" watchObservedRunningTime="2026-07-17 05:41:22.135301664 +0000 UTC m=+4.577240244"
Jul 17 05:41:23.072898 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:23.072866    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:23.073379 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:23.072995    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:23.152301 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:23.152259    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal" event={"ID":"5aa1ec63b2d121d5f010fce817a84937","Type":"ContainerStarted","Data":"5eb5209753c4f3887a48f56e23f108170a998a931d3e287006db44abf4af89d7"}
Jul 17 05:41:23.169686 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:23.168481    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-96.ec2.internal" podStartSLOduration=3.168461075 podStartE2EDuration="3.168461075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:41:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:23.168342706 +0000 UTC m=+5.610281284" watchObservedRunningTime="2026-07-17 05:41:23.168461075 +0000 UTC m=+5.610399645"
Jul 17 05:41:23.552567 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:23.551927    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:23.552567 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:23.552076    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:23.552567 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:23.552142    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret podName:1b659b41-1e15-4c23-95b6-0396eadae285 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:27.552122986 +0000 UTC m=+9.994061553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret") pod "global-pull-secret-syncer-h2lgh" (UID: "1b659b41-1e15-4c23-95b6-0396eadae285") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:23.753852 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:23.753809    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:23.754036 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:23.753920    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq2z4\" (UniqueName: \"kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4\") pod \"network-check-target-85qp6\" (UID: \"b24412fb-094a-4e0f-8de0-b848ebaf4323\") " pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:23.754112 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:23.754074    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:41:23.754112 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:23.754095    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:41:23.754112 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:23.754108    2558 projected.go:196] Error preparing data for projected volume kube-api-access-sq2z4 for pod openshift-network-diagnostics/network-check-target-85qp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:23.754267 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:23.754172    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4 podName:b24412fb-094a-4e0f-8de0-b848ebaf4323 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:27.754152415 +0000 UTC m=+10.196090987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sq2z4" (UniqueName: "kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4") pod "network-check-target-85qp6" (UID: "b24412fb-094a-4e0f-8de0-b848ebaf4323") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:23.754267 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:23.754263    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:23.754380 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:23.754305    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs podName:e92b27e5-e21d-46ed-8a43-212e4f441cae nodeName:}" failed. No retries permitted until 2026-07-17 05:41:27.754292749 +0000 UTC m=+10.196231309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs") pod "network-metrics-daemon-tnctb" (UID: "e92b27e5-e21d-46ed-8a43-212e4f441cae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:24.072909 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:24.072879    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:24.073383 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:24.073054    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:24.073586 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:24.073466    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:24.073680 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:24.073641    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:25.072531 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:25.072474    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:25.072753 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:25.072636    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:26.075093 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:26.074579    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:26.075093 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:26.074720    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:26.075093 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:26.074767    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:26.075093 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:26.074900    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:27.072811 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:27.072294    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:27.072811 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:27.072426    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:27.589184 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:27.589065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:27.589660 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:27.589257    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:27.589660 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:27.589338    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret podName:1b659b41-1e15-4c23-95b6-0396eadae285 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:35.589316688 +0000 UTC m=+18.031255264 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret") pod "global-pull-secret-syncer-h2lgh" (UID: "1b659b41-1e15-4c23-95b6-0396eadae285") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:27.790771 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:27.790734    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq2z4\" (UniqueName: \"kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4\") pod \"network-check-target-85qp6\" (UID: \"b24412fb-094a-4e0f-8de0-b848ebaf4323\") " pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:27.790969 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:27.790786    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:27.790969 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:27.790890    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:27.790969 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:27.790945    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs podName:e92b27e5-e21d-46ed-8a43-212e4f441cae nodeName:}" failed. No retries permitted until 2026-07-17 05:41:35.790930214 +0000 UTC m=+18.232868767 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs") pod "network-metrics-daemon-tnctb" (UID: "e92b27e5-e21d-46ed-8a43-212e4f441cae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:27.791311 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:27.791286    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:41:27.791311 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:27.791305    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:41:27.791311 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:27.791315    2558 projected.go:196] Error preparing data for projected volume kube-api-access-sq2z4 for pod openshift-network-diagnostics/network-check-target-85qp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:27.791498 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:27.791349    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4 podName:b24412fb-094a-4e0f-8de0-b848ebaf4323 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:35.791338053 +0000 UTC m=+18.233276607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sq2z4" (UniqueName: "kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4") pod "network-check-target-85qp6" (UID: "b24412fb-094a-4e0f-8de0-b848ebaf4323") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:28.075962 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:28.075297    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:28.075962 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:28.075411    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:28.075962 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:28.075770    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:28.075962 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:28.075854    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:29.072732 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:29.072553    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:29.078289 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:29.072692    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:30.072933 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:30.072632    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:30.072933 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:30.072672    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:30.072933 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:30.072773    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:30.073627 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:30.073591    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:31.072846 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:31.072814    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:31.073003 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:31.072921    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:32.072173 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:32.072145    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:32.072283 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:32.072240    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:32.072345 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:32.072319    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:32.072420 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:32.072399    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:33.072268 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:33.072236    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:33.072735 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:33.072356    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:34.072640 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:34.072601    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:34.073017 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:34.072602    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:34.073017 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:34.072716    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:34.073017 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:34.072820    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:35.072434 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:35.072401    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:35.072643 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:35.072525    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:35.649270 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:35.649231    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:35.649719 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:35.649360    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:35.649719 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:35.649428    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret podName:1b659b41-1e15-4c23-95b6-0396eadae285 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:51.649410091 +0000 UTC m=+34.091348645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret") pod "global-pull-secret-syncer-h2lgh" (UID: "1b659b41-1e15-4c23-95b6-0396eadae285") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:35.850556 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:35.850522    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq2z4\" (UniqueName: \"kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4\") pod \"network-check-target-85qp6\" (UID: \"b24412fb-094a-4e0f-8de0-b848ebaf4323\") " pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:35.850730 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:35.850571    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:35.850730 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:35.850711    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:41:35.850730 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:35.850727    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:35.850857 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:35.850736    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:41:35.850857 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:35.850749    2558 projected.go:196] Error preparing data for projected volume kube-api-access-sq2z4 for pod openshift-network-diagnostics/network-check-target-85qp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:35.850857 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:35.850788    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs podName:e92b27e5-e21d-46ed-8a43-212e4f441cae nodeName:}" failed. No retries permitted until 2026-07-17 05:41:51.850769326 +0000 UTC m=+34.292707895 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs") pod "network-metrics-daemon-tnctb" (UID: "e92b27e5-e21d-46ed-8a43-212e4f441cae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:35.850857 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:35.850807    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4 podName:b24412fb-094a-4e0f-8de0-b848ebaf4323 nodeName:}" failed. No retries permitted until 2026-07-17 05:41:51.850797392 +0000 UTC m=+34.292735949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sq2z4" (UniqueName: "kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4") pod "network-check-target-85qp6" (UID: "b24412fb-094a-4e0f-8de0-b848ebaf4323") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:36.072535 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:36.072445    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:36.072535 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:36.072445    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:36.072791 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:36.072593    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:36.072791 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:36.072723    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:37.072511 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:37.072459    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:37.072938 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:37.072605    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:38.075984 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:38.075942    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:38.075984 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:38.075967    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:38.076461 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:38.076072    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:38.076461 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:38.076128    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:39.072425 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:39.072389    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:39.072610 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:39.072515    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:40.072817 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.072672    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:40.073309 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.072727    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:40.073309 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:40.072897    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:40.073309 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:40.072958    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:40.185685 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.185650    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl" event={"ID":"44bef013-3171-42a3-8771-13dda23de637","Type":"ContainerStarted","Data":"904f3ec40960e185310876c8f0344c19315114ecfa643bcccf1d3d2889efb74d"}
Jul 17 05:41:40.187464 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.187145    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6drsh" event={"ID":"19e2915f-ebfc-45ff-9452-9e0c35a105ca","Type":"ContainerStarted","Data":"2b41d5f0fabfbba5fa69d29e332582d581a16893d50d83d81b51208a223582d0"}
Jul 17 05:41:40.189043 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.189025    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 05:41:40.189591 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.189566    2558 generic.go:356] "Generic (PLEG): container finished" podID="56e41793-4279-46d8-84a0-ad435e812001" containerID="240827380ee8b160e95ac8a101c0ed8adbe5f92f6b67ca64ef124e478928c6c7" exitCode=1
Jul 17 05:41:40.189668 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.189629    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8" event={"ID":"56e41793-4279-46d8-84a0-ad435e812001","Type":"ContainerDied","Data":"240827380ee8b160e95ac8a101c0ed8adbe5f92f6b67ca64ef124e478928c6c7"}
Jul 17 05:41:40.189714 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.189701    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8" event={"ID":"56e41793-4279-46d8-84a0-ad435e812001","Type":"ContainerStarted","Data":"2987e94cf7fc396f72594c087cabc81b04eefb4d5f3e415a76123248e2f8907c"}
Jul 17 05:41:40.191273 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.191254    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lfzc2" event={"ID":"2a31ea27-5883-4df9-9032-72ad9c0ee4ea","Type":"ContainerStarted","Data":"4080223525f92063ca0eccd2b4d4630bf2e076e1b6aac8dfe35de0d2bc84d977"}
Jul 17 05:41:40.192996 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.192969    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-crk4m" event={"ID":"5a773d59-6bb2-4da5-97e6-53ae51a55cd7","Type":"ContainerStarted","Data":"c7b0da1d8b263d49e316f3c72c476debd6fb2320e3627b0cd7d6412f7d6eabe6"}
Jul 17 05:41:40.194363 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.194341    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rflw8" event={"ID":"9c14d858-96a0-4454-9f78-f2a403da7d11","Type":"ContainerStarted","Data":"8ee69a310caeb64b84af27d193c2f7e362d921a33172f1e12755e8ab5faea57c"}
Jul 17 05:41:40.196009 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.195984    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-72jjp" event={"ID":"90b3c468-7bbf-41b8-9a4a-7922b74b2d47","Type":"ContainerStarted","Data":"832e27f8d1e60e122f79dec5ba2b7b1ba444658383b06d7c0420031ea3752861"}
Jul 17 05:41:40.197344 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.197327    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jtnfh" event={"ID":"5626b442-c1f4-4444-92d8-fb67270cab22","Type":"ContainerStarted","Data":"bce3e11326f77ee1db8a6372a3eb970dc3d6bacc8a314a6f265aab20e20617be"}
Jul 17 05:41:40.219722 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.219677    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lfzc2" podStartSLOduration=3.475257193 podStartE2EDuration="22.21966186s" totalImagesPullingTime="18.744404667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:40.21941277 +0000 UTC m=+22.661351343" watchObservedRunningTime="2026-07-17 05:41:40.21966186 +0000 UTC m=+22.661600439"
Jul 17 05:41:40.219835 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.219754    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6drsh" podStartSLOduration=3.489958797 podStartE2EDuration="22.219748498s" totalImagesPullingTime="18.729789701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:40.20470458 +0000 UTC m=+22.646643155" watchObservedRunningTime="2026-07-17 05:41:40.219748498 +0000 UTC m=+22.661687070"
Jul 17 05:41:40.237167 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.237128    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-crk4m" podStartSLOduration=11.25650912 podStartE2EDuration="22.237114949s" totalImagesPullingTime="10.980605829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:40.236798621 +0000 UTC m=+22.678737207" watchObservedRunningTime="2026-07-17 05:41:40.237114949 +0000 UTC m=+22.679053523"
Jul 17 05:41:40.255846 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.255807    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-72jjp" podStartSLOduration=3.490116301 podStartE2EDuration="22.255792386s" totalImagesPullingTime="18.765676085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:40.255240551 +0000 UTC m=+22.697179124" watchObservedRunningTime="2026-07-17 05:41:40.255792386 +0000 UTC m=+22.697730959"
Jul 17 05:41:40.291240 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:40.291199    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jtnfh" podStartSLOduration=3.484680096 podStartE2EDuration="22.29118426s" totalImagesPullingTime="18.806504164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:40.290790067 +0000 UTC m=+22.732728641" watchObservedRunningTime="2026-07-17 05:41:40.29118426 +0000 UTC m=+22.733122836"
Jul 17 05:41:41.072692 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.072537    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:41.072776 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:41.072754    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:41.203962 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.203943    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 05:41:41.204390 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.204365    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8" event={"ID":"56e41793-4279-46d8-84a0-ad435e812001","Type":"ContainerStarted","Data":"c31969bb71ed336c9e939865ec62e65c5104508380e8b12e268373e8c75a9e41"}
Jul 17 05:41:41.204461 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.204446    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8" event={"ID":"56e41793-4279-46d8-84a0-ad435e812001","Type":"ContainerStarted","Data":"85e2c6479a393e1b9d97dbf9fc91dfa273d1a291c752f88fc326b6adff7c35be"}
Jul 17 05:41:41.204542 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.204525    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8" event={"ID":"56e41793-4279-46d8-84a0-ad435e812001","Type":"ContainerStarted","Data":"0ded4d90a84c7e31e1607f3d7d36320444101c1d24c9d94802a03dde67241cc6"}
Jul 17 05:41:41.204596 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.204573    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8" event={"ID":"56e41793-4279-46d8-84a0-ad435e812001","Type":"ContainerStarted","Data":"85504ebe87b5badc1f4234a1a5fc41ee14ef269839225e863b5e5dec6b1285d1"}
Jul 17 05:41:41.206060 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.206030    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vzmnt" event={"ID":"2fbb5583-456e-4c0a-b9e8-af3dd2c40797","Type":"ContainerStarted","Data":"3599a3b4264e92a78c7d00c248f8fd985f80225b45275bb7f447b938b624795e"}
Jul 17 05:41:41.211451 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.211429    2558 generic.go:356] "Generic (PLEG): container finished" podID="9c14d858-96a0-4454-9f78-f2a403da7d11" containerID="8ee69a310caeb64b84af27d193c2f7e362d921a33172f1e12755e8ab5faea57c" exitCode=0
Jul 17 05:41:41.211566 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.211520    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rflw8" event={"ID":"9c14d858-96a0-4454-9f78-f2a403da7d11","Type":"ContainerDied","Data":"8ee69a310caeb64b84af27d193c2f7e362d921a33172f1e12755e8ab5faea57c"}
Jul 17 05:41:41.222890 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.222849    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vzmnt" podStartSLOduration=4.49230538 podStartE2EDuration="23.222833329s" totalImagesPullingTime="18.730527949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:41.222651161 +0000 UTC m=+23.664589745" watchObservedRunningTime="2026-07-17 05:41:41.222833329 +0000 UTC m=+23.664771904"
Jul 17 05:41:41.301264 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.301208    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 05:41:41.598936 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.598882    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:41.599355 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.599341    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:41.962327 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.962173    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T05:41:41.301229131Z","UUID":"04647e3e-e814-4a75-ae0f-1ae2fd6dbc6f","Handler":null,"Name":"","Endpoint":""}
Jul 17 05:41:41.964955 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.964582    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 05:41:41.964955 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:41.964623    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 05:41:42.072494 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:42.072440    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:42.072674 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:42.072444    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:42.072674 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:42.072612    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:42.072833 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:42.072751    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:42.216395 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:42.216308    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl" event={"ID":"44bef013-3171-42a3-8771-13dda23de637","Type":"ContainerStarted","Data":"e157221af1345ef4fba4ffcd3c1913a7b4131a6c31d2ce4f5d5b14cd085a2768"}
Jul 17 05:41:42.216860 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:42.216659    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:42.217185 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:42.217162    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-crk4m"
Jul 17 05:41:43.072766 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:43.072544    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:43.072947 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:43.072844    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:43.222016 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:43.221509    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl" event={"ID":"44bef013-3171-42a3-8771-13dda23de637","Type":"ContainerStarted","Data":"921cf9c2e8c5b4a49ed8ce6d02ca39d35a79317c8a45e986981824edaaea18d2"}
Jul 17 05:41:43.225855 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:43.225822    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 05:41:43.228926 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:43.228891    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8" event={"ID":"56e41793-4279-46d8-84a0-ad435e812001","Type":"ContainerStarted","Data":"df3adc7c9d8beaff08a9177aec3c065111cc3be37485a4bf668d19b0bd192df3"}
Jul 17 05:41:43.239561 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:43.239524    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hhxcl" podStartSLOduration=4.19153152 podStartE2EDuration="25.239511194s" totalImagesPullingTime="21.047979674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:43.239400113 +0000 UTC m=+25.681338682" watchObservedRunningTime="2026-07-17 05:41:43.239511194 +0000 UTC m=+25.681449767"
Jul 17 05:41:44.072781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:44.072746    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:44.072978 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:44.072872    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:44.073223 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:44.073209    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:44.073326 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:44.073307    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:45.072773 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:45.072692    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:45.073304 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:45.072817    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:46.073007 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:46.072776    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:46.073510 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:46.072783    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:46.073510 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:46.073044    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:46.073510 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:46.073124    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:46.239309 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:46.239278    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 05:41:46.239679 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:46.239651    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8" event={"ID":"56e41793-4279-46d8-84a0-ad435e812001","Type":"ContainerStarted","Data":"03697e6bd0b81177c933f26f28a0962aedd7d6547b1f71d4a4bfd3143baead3c"}
Jul 17 05:41:46.239896 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:46.239879    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:46.240090 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:46.240067    2558 scope.go:117] "RemoveContainer" containerID="240827380ee8b160e95ac8a101c0ed8adbe5f92f6b67ca64ef124e478928c6c7"
Jul 17 05:41:46.241581 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:46.241555    2558 generic.go:356] "Generic (PLEG): container finished" podID="9c14d858-96a0-4454-9f78-f2a403da7d11" containerID="76f9696e9b53af00f7e2d3d04a827491b4ea8c9bea28b5493a6a26a2266389b4" exitCode=0
Jul 17 05:41:46.241689 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:46.241591    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rflw8" event={"ID":"9c14d858-96a0-4454-9f78-f2a403da7d11","Type":"ContainerDied","Data":"76f9696e9b53af00f7e2d3d04a827491b4ea8c9bea28b5493a6a26a2266389b4"}
Jul 17 05:41:46.256690 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:46.256665    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:47.072394 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.072363    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:47.072628 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:47.072497    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:47.248061 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.248037    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 05:41:47.248595 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.248439    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8" event={"ID":"56e41793-4279-46d8-84a0-ad435e812001","Type":"ContainerStarted","Data":"d63aa24ddd1c864ef6ff13791413d57e761e2c8c8b7f639b34e2daeaf87056b3"}
Jul 17 05:41:47.248744 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.248724    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:47.248911 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.248893    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:47.263881 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.263856    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:41:47.276566 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.276516    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8" podStartSLOduration=10.466297659 podStartE2EDuration="29.276500667s" totalImagesPullingTime="18.810203008s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:47.275737682 +0000 UTC m=+29.717676274" watchObservedRunningTime="2026-07-17 05:41:47.276500667 +0000 UTC m=+29.718439241"
Jul 17 05:41:47.724180 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.724107    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tnctb"]
Jul 17 05:41:47.724301 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.724216    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:47.724341 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:47.724300    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:47.724632 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.724615    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-85qp6"]
Jul 17 05:41:47.724694 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.724684    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:47.724763 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:47.724749    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:47.735611 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.735589    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h2lgh"]
Jul 17 05:41:47.735705 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:47.735665    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:47.735746 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:47.735727    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:48.252379 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:48.252161    2558 generic.go:356] "Generic (PLEG): container finished" podID="9c14d858-96a0-4454-9f78-f2a403da7d11" containerID="552565e599edba4d405aa254292b388d61cd07a6fd28196219bcf0dcb1b014cb" exitCode=0
Jul 17 05:41:48.252765 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:48.252242    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rflw8" event={"ID":"9c14d858-96a0-4454-9f78-f2a403da7d11","Type":"ContainerDied","Data":"552565e599edba4d405aa254292b388d61cd07a6fd28196219bcf0dcb1b014cb"}
Jul 17 05:41:49.072798 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:49.072723    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:49.072798 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:49.072771    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:49.073034 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:49.072853    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:49.073034 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:49.072961    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:49.257401 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:49.257373    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rflw8" event={"ID":"9c14d858-96a0-4454-9f78-f2a403da7d11","Type":"ContainerStarted","Data":"e39fdf4491748dbedb257d83138a3747209bc16da095975cca279e8bb685c735"}
Jul 17 05:41:50.072421 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:50.072355    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:50.072564 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:50.072447    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:50.261968 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:50.261936    2558 generic.go:356] "Generic (PLEG): container finished" podID="9c14d858-96a0-4454-9f78-f2a403da7d11" containerID="e39fdf4491748dbedb257d83138a3747209bc16da095975cca279e8bb685c735" exitCode=0
Jul 17 05:41:50.262457 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:50.261972    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rflw8" event={"ID":"9c14d858-96a0-4454-9f78-f2a403da7d11","Type":"ContainerDied","Data":"e39fdf4491748dbedb257d83138a3747209bc16da095975cca279e8bb685c735"}
Jul 17 05:41:51.072141 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:51.072107    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:51.072294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:51.072107    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:51.072294 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:51.072244    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:41:51.072413 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:51.072314    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-85qp6" podUID="b24412fb-094a-4e0f-8de0-b848ebaf4323"
Jul 17 05:41:51.665837 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:51.665805    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:51.666288 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:51.665991    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:51.666288 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:51.666067    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret podName:1b659b41-1e15-4c23-95b6-0396eadae285 nodeName:}" failed. No retries permitted until 2026-07-17 05:42:23.66604646 +0000 UTC m=+66.107985017 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret") pod "global-pull-secret-syncer-h2lgh" (UID: "1b659b41-1e15-4c23-95b6-0396eadae285") : object "kube-system"/"original-pull-secret" not registered
Jul 17 05:41:51.867691 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:51.867657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq2z4\" (UniqueName: \"kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4\") pod \"network-check-target-85qp6\" (UID: \"b24412fb-094a-4e0f-8de0-b848ebaf4323\") " pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:51.867889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:51.867719    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:51.867889 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:51.867824    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 05:41:51.867889 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:51.867837    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:51.867889 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:51.867852    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 05:41:51.867889 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:51.867865    2558 projected.go:196] Error preparing data for projected volume kube-api-access-sq2z4 for pod openshift-network-diagnostics/network-check-target-85qp6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:51.868144 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:51.867904    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs podName:e92b27e5-e21d-46ed-8a43-212e4f441cae nodeName:}" failed. No retries permitted until 2026-07-17 05:42:23.867882341 +0000 UTC m=+66.309820906 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs") pod "network-metrics-daemon-tnctb" (UID: "e92b27e5-e21d-46ed-8a43-212e4f441cae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 05:41:51.868144 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:51.867918    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4 podName:b24412fb-094a-4e0f-8de0-b848ebaf4323 nodeName:}" failed. No retries permitted until 2026-07-17 05:42:23.867911607 +0000 UTC m=+66.309850159 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sq2z4" (UniqueName: "kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4") pod "network-check-target-85qp6" (UID: "b24412fb-094a-4e0f-8de0-b848ebaf4323") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 05:41:52.072992 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.072915    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:52.073135 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:52.073053    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2lgh" podUID="1b659b41-1e15-4c23-95b6-0396eadae285"
Jul 17 05:41:52.860035 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.860001    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-96.ec2.internal" event="NodeReady"
Jul 17 05:41:52.860414 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.860143    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 05:41:52.904237 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.903524    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9kcb2"]
Jul 17 05:41:52.906198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.906178    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:52.906691 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.906668    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dwxmp"]
Jul 17 05:41:52.908667 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.908647    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dwxmp"
Jul 17 05:41:52.908851 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.908828    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jx7lr\""
Jul 17 05:41:52.908975 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.908881    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 05:41:52.909180 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.909164    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 05:41:52.911361 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.911187    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 05:41:52.911432 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.911353    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 05:41:52.911728 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.911711    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tmkvs\""
Jul 17 05:41:52.911813 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.911739    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 05:41:52.916181 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.916152    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9kcb2"]
Jul 17 05:41:52.918818 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:52.918798    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dwxmp"]
Jul 17 05:41:53.072150 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.072093    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:41:53.072150 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.072128    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:41:53.075639 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.075328    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 05:41:53.075639 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.075343    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 05:41:53.075639 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.075348    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9jxzr\""
Jul 17 05:41:53.075639 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.075352    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7x5wz\""
Jul 17 05:41:53.075639 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.075365    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 05:41:53.075922 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.075649    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rl86\" (UniqueName: \"kubernetes.io/projected/480c28d5-a4e5-4bef-9605-a0ceff3cc4b6-kube-api-access-6rl86\") pod \"ingress-canary-dwxmp\" (UID: \"480c28d5-a4e5-4bef-9605-a0ceff3cc4b6\") " pod="openshift-ingress-canary/ingress-canary-dwxmp"
Jul 17 05:41:53.075922 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.075686    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.075922 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.075703    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce41f40b-970f-4698-aa72-e7f772641eaf-tmp-dir\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.075922 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.075805    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce41f40b-970f-4698-aa72-e7f772641eaf-config-volume\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.075922 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.075843    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkzc9\" (UniqueName: \"kubernetes.io/projected/ce41f40b-970f-4698-aa72-e7f772641eaf-kube-api-access-hkzc9\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.075922 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.075869    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/480c28d5-a4e5-4bef-9605-a0ceff3cc4b6-cert\") pod \"ingress-canary-dwxmp\" (UID: \"480c28d5-a4e5-4bef-9605-a0ceff3cc4b6\") " pod="openshift-ingress-canary/ingress-canary-dwxmp"
Jul 17 05:41:53.176424 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.176356    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce41f40b-970f-4698-aa72-e7f772641eaf-config-volume\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.176424 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.176394    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkzc9\" (UniqueName: \"kubernetes.io/projected/ce41f40b-970f-4698-aa72-e7f772641eaf-kube-api-access-hkzc9\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.176424 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.176418    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/480c28d5-a4e5-4bef-9605-a0ceff3cc4b6-cert\") pod \"ingress-canary-dwxmp\" (UID: \"480c28d5-a4e5-4bef-9605-a0ceff3cc4b6\") " pod="openshift-ingress-canary/ingress-canary-dwxmp"
Jul 17 05:41:53.176637 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.176455    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rl86\" (UniqueName: \"kubernetes.io/projected/480c28d5-a4e5-4bef-9605-a0ceff3cc4b6-kube-api-access-6rl86\") pod \"ingress-canary-dwxmp\" (UID: \"480c28d5-a4e5-4bef-9605-a0ceff3cc4b6\") " pod="openshift-ingress-canary/ingress-canary-dwxmp"
Jul 17 05:41:53.176637 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.176514    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.176637 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.176534    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce41f40b-970f-4698-aa72-e7f772641eaf-tmp-dir\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.176822 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.176800    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce41f40b-970f-4698-aa72-e7f772641eaf-tmp-dir\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.177265 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.177246    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce41f40b-970f-4698-aa72-e7f772641eaf-config-volume\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.177508 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:53.177489    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:41:53.177566 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:53.177558    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls podName:ce41f40b-970f-4698-aa72-e7f772641eaf nodeName:}" failed. No retries permitted until 2026-07-17 05:41:53.677541429 +0000 UTC m=+36.119479982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls") pod "dns-default-9kcb2" (UID: "ce41f40b-970f-4698-aa72-e7f772641eaf") : secret "dns-default-metrics-tls" not found
Jul 17 05:41:53.181565 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.181549    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/480c28d5-a4e5-4bef-9605-a0ceff3cc4b6-cert\") pod \"ingress-canary-dwxmp\" (UID: \"480c28d5-a4e5-4bef-9605-a0ceff3cc4b6\") " pod="openshift-ingress-canary/ingress-canary-dwxmp"
Jul 17 05:41:53.188522 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.188499    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkzc9\" (UniqueName: \"kubernetes.io/projected/ce41f40b-970f-4698-aa72-e7f772641eaf-kube-api-access-hkzc9\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.188642 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.188622    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rl86\" (UniqueName: \"kubernetes.io/projected/480c28d5-a4e5-4bef-9605-a0ceff3cc4b6-kube-api-access-6rl86\") pod \"ingress-canary-dwxmp\" (UID: \"480c28d5-a4e5-4bef-9605-a0ceff3cc4b6\") " pod="openshift-ingress-canary/ingress-canary-dwxmp"
Jul 17 05:41:53.224549 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.224530    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dwxmp"
Jul 17 05:41:53.679398 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:53.679357    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:53.679592 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:53.679552    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:41:53.679649 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:53.679642    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls podName:ce41f40b-970f-4698-aa72-e7f772641eaf nodeName:}" failed. No retries permitted until 2026-07-17 05:41:54.679619333 +0000 UTC m=+37.121557902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls") pod "dns-default-9kcb2" (UID: "ce41f40b-970f-4698-aa72-e7f772641eaf") : secret "dns-default-metrics-tls" not found
Jul 17 05:41:54.072344 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:54.072273    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:41:54.075273 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:54.075250    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 05:41:54.264985 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:54.264740    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dwxmp"]
Jul 17 05:41:54.268852 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:41:54.268823    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod480c28d5_a4e5_4bef_9605_a0ceff3cc4b6.slice/crio-df94e7b5338de9630874a551e556238b8300ac62076d804a76e6e57f6f8ea3ec WatchSource:0}: Error finding container df94e7b5338de9630874a551e556238b8300ac62076d804a76e6e57f6f8ea3ec: Status 404 returned error can't find the container with id df94e7b5338de9630874a551e556238b8300ac62076d804a76e6e57f6f8ea3ec
Jul 17 05:41:54.274983 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:54.274932    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rflw8" event={"ID":"9c14d858-96a0-4454-9f78-f2a403da7d11","Type":"ContainerStarted","Data":"084ad6ef512ad8686d15da37432863653fd7c1cd5f19c4ed3f983de76eb4c87b"}
Jul 17 05:41:54.276119 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:54.276099    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dwxmp" event={"ID":"480c28d5-a4e5-4bef-9605-a0ceff3cc4b6","Type":"ContainerStarted","Data":"df94e7b5338de9630874a551e556238b8300ac62076d804a76e6e57f6f8ea3ec"}
Jul 17 05:41:54.688040 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:54.688006    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:54.688162 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:54.688142    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:41:54.688223 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:54.688202    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls podName:ce41f40b-970f-4698-aa72-e7f772641eaf nodeName:}" failed. No retries permitted until 2026-07-17 05:41:56.688187587 +0000 UTC m=+39.130126140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls") pod "dns-default-9kcb2" (UID: "ce41f40b-970f-4698-aa72-e7f772641eaf") : secret "dns-default-metrics-tls" not found
Jul 17 05:41:55.282144 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:55.282107    2558 generic.go:356] "Generic (PLEG): container finished" podID="9c14d858-96a0-4454-9f78-f2a403da7d11" containerID="084ad6ef512ad8686d15da37432863653fd7c1cd5f19c4ed3f983de76eb4c87b" exitCode=0
Jul 17 05:41:55.282585 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:55.282160    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rflw8" event={"ID":"9c14d858-96a0-4454-9f78-f2a403da7d11","Type":"ContainerDied","Data":"084ad6ef512ad8686d15da37432863653fd7c1cd5f19c4ed3f983de76eb4c87b"}
Jul 17 05:41:56.287517 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:56.287466    2558 generic.go:356] "Generic (PLEG): container finished" podID="9c14d858-96a0-4454-9f78-f2a403da7d11" containerID="97a6d62330756948bfeeaee3192f5a7756b152ffd1359d4f7252bc6d60c9fb91" exitCode=0
Jul 17 05:41:56.287517 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:56.287504    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rflw8" event={"ID":"9c14d858-96a0-4454-9f78-f2a403da7d11","Type":"ContainerDied","Data":"97a6d62330756948bfeeaee3192f5a7756b152ffd1359d4f7252bc6d60c9fb91"}
Jul 17 05:41:56.703087 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:56.702916    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:41:56.703220 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:56.703065    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:41:56.703220 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:41:56.703174    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls podName:ce41f40b-970f-4698-aa72-e7f772641eaf nodeName:}" failed. No retries permitted until 2026-07-17 05:42:00.703158763 +0000 UTC m=+43.145097315 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls") pod "dns-default-9kcb2" (UID: "ce41f40b-970f-4698-aa72-e7f772641eaf") : secret "dns-default-metrics-tls" not found
Jul 17 05:41:57.293386 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:57.293358    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rflw8" event={"ID":"9c14d858-96a0-4454-9f78-f2a403da7d11","Type":"ContainerStarted","Data":"c00f831f04f9607e14aeb3e6e90dbdcf134e6348efab9a39f30d42acf4d14df9"}
Jul 17 05:41:57.294786 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:57.294760    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dwxmp" event={"ID":"480c28d5-a4e5-4bef-9605-a0ceff3cc4b6","Type":"ContainerStarted","Data":"59933929c27bb9115fe58300e2eec0bb20be9a889757f1a3505f7f169eba1dcc"}
Jul 17 05:41:57.315254 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:57.315207    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rflw8" podStartSLOduration=10.186009024 podStartE2EDuration="39.315193281s" totalImagesPullingTime="29.129184257s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:57.313734826 +0000 UTC m=+39.755673399" watchObservedRunningTime="2026-07-17 05:41:57.315193281 +0000 UTC m=+39.757131854"
Jul 17 05:41:57.328430 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:41:57.328393    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dwxmp" podStartSLOduration=3.17286592 podStartE2EDuration="5.328382504s" totalImagesPullingTime="2.155516584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:57.327766218 +0000 UTC m=+39.769704791" watchObservedRunningTime="2026-07-17 05:41:57.328382504 +0000 UTC m=+39.770321076"
Jul 17 05:42:00.729506 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:00.729459    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:42:00.729960 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:42:00.729587    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:42:00.729960 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:42:00.729641    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls podName:ce41f40b-970f-4698-aa72-e7f772641eaf nodeName:}" failed. No retries permitted until 2026-07-17 05:42:08.729625715 +0000 UTC m=+51.171564267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls") pod "dns-default-9kcb2" (UID: "ce41f40b-970f-4698-aa72-e7f772641eaf") : secret "dns-default-metrics-tls" not found
Jul 17 05:42:08.780948 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:08.780918    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:42:08.781341 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:42:08.781055    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:42:08.781341 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:42:08.781109    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls podName:ce41f40b-970f-4698-aa72-e7f772641eaf nodeName:}" failed. No retries permitted until 2026-07-17 05:42:24.781095924 +0000 UTC m=+67.223034476 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls") pod "dns-default-9kcb2" (UID: "ce41f40b-970f-4698-aa72-e7f772641eaf") : secret "dns-default-metrics-tls" not found
Jul 17 05:42:19.273134 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:19.273108    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fj8r8"
Jul 17 05:42:20.399680 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.399650    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"]
Jul 17 05:42:20.402265 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.402248    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:20.404964 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.404945    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 17 05:42:20.406346 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.406328    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 17 05:42:20.406401 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.406348    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 17 05:42:20.406401 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.406370    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 17 05:42:20.411696 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.411672    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"]
Jul 17 05:42:20.558444 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.558419    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69a662f9-bf73-47d4-9687-34b7b6b41769-tmp\") pod \"klusterlet-addon-workmgr-d54864cb-pjqwc\" (UID: \"69a662f9-bf73-47d4-9687-34b7b6b41769\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:20.558444 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.558443    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4t4t\" (UniqueName: \"kubernetes.io/projected/69a662f9-bf73-47d4-9687-34b7b6b41769-kube-api-access-f4t4t\") pod \"klusterlet-addon-workmgr-d54864cb-pjqwc\" (UID: \"69a662f9-bf73-47d4-9687-34b7b6b41769\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:20.558590 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.558487    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/69a662f9-bf73-47d4-9687-34b7b6b41769-klusterlet-config\") pod \"klusterlet-addon-workmgr-d54864cb-pjqwc\" (UID: \"69a662f9-bf73-47d4-9687-34b7b6b41769\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:20.659702 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.659654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69a662f9-bf73-47d4-9687-34b7b6b41769-tmp\") pod \"klusterlet-addon-workmgr-d54864cb-pjqwc\" (UID: \"69a662f9-bf73-47d4-9687-34b7b6b41769\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:20.659702 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.659680    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4t4t\" (UniqueName: \"kubernetes.io/projected/69a662f9-bf73-47d4-9687-34b7b6b41769-kube-api-access-f4t4t\") pod \"klusterlet-addon-workmgr-d54864cb-pjqwc\" (UID: \"69a662f9-bf73-47d4-9687-34b7b6b41769\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:20.659806 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.659712    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/69a662f9-bf73-47d4-9687-34b7b6b41769-klusterlet-config\") pod \"klusterlet-addon-workmgr-d54864cb-pjqwc\" (UID: \"69a662f9-bf73-47d4-9687-34b7b6b41769\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:20.659990 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.659972    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69a662f9-bf73-47d4-9687-34b7b6b41769-tmp\") pod \"klusterlet-addon-workmgr-d54864cb-pjqwc\" (UID: \"69a662f9-bf73-47d4-9687-34b7b6b41769\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:20.663637 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.663622    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/69a662f9-bf73-47d4-9687-34b7b6b41769-klusterlet-config\") pod \"klusterlet-addon-workmgr-d54864cb-pjqwc\" (UID: \"69a662f9-bf73-47d4-9687-34b7b6b41769\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:20.667926 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.667904    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4t4t\" (UniqueName: \"kubernetes.io/projected/69a662f9-bf73-47d4-9687-34b7b6b41769-kube-api-access-f4t4t\") pod \"klusterlet-addon-workmgr-d54864cb-pjqwc\" (UID: \"69a662f9-bf73-47d4-9687-34b7b6b41769\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:20.710927 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.710904    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:20.825624 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:20.825603    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"]
Jul 17 05:42:20.828030 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:42:20.828006    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69a662f9_bf73_47d4_9687_34b7b6b41769.slice/crio-5e09f851edf3ee26c31ed6919692b05ffb0a20d8488b71c656735a852cb5db51 WatchSource:0}: Error finding container 5e09f851edf3ee26c31ed6919692b05ffb0a20d8488b71c656735a852cb5db51: Status 404 returned error can't find the container with id 5e09f851edf3ee26c31ed6919692b05ffb0a20d8488b71c656735a852cb5db51
Jul 17 05:42:21.346281 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:21.346252    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc" event={"ID":"69a662f9-bf73-47d4-9687-34b7b6b41769","Type":"ContainerStarted","Data":"5e09f851edf3ee26c31ed6919692b05ffb0a20d8488b71c656735a852cb5db51"}
Jul 17 05:42:23.684761 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:23.684717    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:42:23.687642 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:23.687619    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 05:42:23.697390 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:23.697367    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b659b41-1e15-4c23-95b6-0396eadae285-original-pull-secret\") pod \"global-pull-secret-syncer-h2lgh\" (UID: \"1b659b41-1e15-4c23-95b6-0396eadae285\") " pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:42:23.782483 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:23.782443    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2lgh"
Jul 17 05:42:23.886484 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:23.886435    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq2z4\" (UniqueName: \"kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4\") pod \"network-check-target-85qp6\" (UID: \"b24412fb-094a-4e0f-8de0-b848ebaf4323\") " pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:42:23.886635 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:23.886496    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:42:23.889264 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:23.889231    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 05:42:23.889401 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:23.889383    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 05:42:23.896853 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:42:23.896830    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 17 05:42:23.896944 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:42:23.896901    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs podName:e92b27e5-e21d-46ed-8a43-212e4f441cae nodeName:}" failed. No retries permitted until 2026-07-17 05:43:27.896880398 +0000 UTC m=+130.338818972 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs") pod "network-metrics-daemon-tnctb" (UID: "e92b27e5-e21d-46ed-8a43-212e4f441cae") : secret "metrics-daemon-secret" not found
Jul 17 05:42:23.899076 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:23.899057    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 05:42:23.910076 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:23.910057    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq2z4\" (UniqueName: \"kubernetes.io/projected/b24412fb-094a-4e0f-8de0-b848ebaf4323-kube-api-access-sq2z4\") pod \"network-check-target-85qp6\" (UID: \"b24412fb-094a-4e0f-8de0-b848ebaf4323\") " pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:42:23.992146 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:23.992089    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9jxzr\""
Jul 17 05:42:24.000204 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:24.000174    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:42:24.588354 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:24.588321    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-85qp6"]
Jul 17 05:42:24.592751 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:42:24.592726    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb24412fb_094a_4e0f_8de0_b848ebaf4323.slice/crio-88e743ee2fe2874ec0ff406bcd76c23d32876db8c89aa38ebeee02dc853ce910 WatchSource:0}: Error finding container 88e743ee2fe2874ec0ff406bcd76c23d32876db8c89aa38ebeee02dc853ce910: Status 404 returned error can't find the container with id 88e743ee2fe2874ec0ff406bcd76c23d32876db8c89aa38ebeee02dc853ce910
Jul 17 05:42:24.604953 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:24.604929    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h2lgh"]
Jul 17 05:42:24.608319 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:42:24.608294    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b659b41_1e15_4c23_95b6_0396eadae285.slice/crio-ef1e1fa6f13d5c1ba0fe9eae971743b3ec533010d2cd1cdcefc0594270b3ff10 WatchSource:0}: Error finding container ef1e1fa6f13d5c1ba0fe9eae971743b3ec533010d2cd1cdcefc0594270b3ff10: Status 404 returned error can't find the container with id ef1e1fa6f13d5c1ba0fe9eae971743b3ec533010d2cd1cdcefc0594270b3ff10
Jul 17 05:42:24.792789 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:24.792726    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:42:24.793052 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:42:24.792875    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:42:24.793052 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:42:24.792933    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls podName:ce41f40b-970f-4698-aa72-e7f772641eaf nodeName:}" failed. No retries permitted until 2026-07-17 05:42:56.792915314 +0000 UTC m=+99.234853882 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls") pod "dns-default-9kcb2" (UID: "ce41f40b-970f-4698-aa72-e7f772641eaf") : secret "dns-default-metrics-tls" not found
Jul 17 05:42:25.358528 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:25.358438    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h2lgh" event={"ID":"1b659b41-1e15-4c23-95b6-0396eadae285","Type":"ContainerStarted","Data":"ef1e1fa6f13d5c1ba0fe9eae971743b3ec533010d2cd1cdcefc0594270b3ff10"}
Jul 17 05:42:25.359903 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:25.359873    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-85qp6" event={"ID":"b24412fb-094a-4e0f-8de0-b848ebaf4323","Type":"ContainerStarted","Data":"88e743ee2fe2874ec0ff406bcd76c23d32876db8c89aa38ebeee02dc853ce910"}
Jul 17 05:42:25.361690 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:25.361636    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc" event={"ID":"69a662f9-bf73-47d4-9687-34b7b6b41769","Type":"ContainerStarted","Data":"5e268ef49afe101da0f40f19a859bdf756dfaf9faa7e15530ae4cf6790739129"}
Jul 17 05:42:25.361894 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:25.361875    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:25.363548 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:25.363525    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:42:25.377030 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:25.376987    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc" podStartSLOduration=1.71643859 podStartE2EDuration="5.376975676s" totalImagesPullingTime="3.660537086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:42:25.376304405 +0000 UTC m=+67.818243002" watchObservedRunningTime="2026-07-17 05:42:25.376975676 +0000 UTC m=+67.818914254"
Jul 17 05:42:29.374000 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:29.373964    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h2lgh" event={"ID":"1b659b41-1e15-4c23-95b6-0396eadae285","Type":"ContainerStarted","Data":"1edf70241e3f88cde78ad7855a80f0db64ce27098b11b1cd4f63ad32d2cb289b"}
Jul 17 05:42:29.375443 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:29.375418    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-85qp6" event={"ID":"b24412fb-094a-4e0f-8de0-b848ebaf4323","Type":"ContainerStarted","Data":"625316dd4baf5d3165f5a8863ea2298529bd8c28cb183bec1998989082d4efb8"}
Jul 17 05:42:29.375558 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:29.375515    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:42:29.389919 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:29.389883    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h2lgh" podStartSLOduration=67.012216875 podStartE2EDuration="1m11.389871965s" totalImagesPullingTime="4.37765509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:42:29.389411226 +0000 UTC m=+71.831349814" watchObservedRunningTime="2026-07-17 05:42:29.389871965 +0000 UTC m=+71.831810537"
Jul 17 05:42:29.403697 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:29.403659    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-85qp6" podStartSLOduration=67.013410413 podStartE2EDuration="1m11.403650875s" totalImagesPullingTime="4.390240462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:42:29.403019523 +0000 UTC m=+71.844958099" watchObservedRunningTime="2026-07-17 05:42:29.403650875 +0000 UTC m=+71.845589448"
Jul 17 05:42:56.800957 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:42:56.800918    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:42:56.801368 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:42:56.801028    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:42:56.801368 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:42:56.801091    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls podName:ce41f40b-970f-4698-aa72-e7f772641eaf nodeName:}" failed. No retries permitted until 2026-07-17 05:44:00.801074185 +0000 UTC m=+163.243012740 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls") pod "dns-default-9kcb2" (UID: "ce41f40b-970f-4698-aa72-e7f772641eaf") : secret "dns-default-metrics-tls" not found
Jul 17 05:43:00.379824 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:43:00.379797    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-85qp6"
Jul 17 05:43:27.992151 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:43:27.992123    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:43:27.992584 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:43:27.992233    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 17 05:43:27.992584 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:43:27.992289    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs podName:e92b27e5-e21d-46ed-8a43-212e4f441cae nodeName:}" failed. No retries permitted until 2026-07-17 05:45:29.992275248 +0000 UTC m=+252.434213800 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs") pod "network-metrics-daemon-tnctb" (UID: "e92b27e5-e21d-46ed-8a43-212e4f441cae") : secret "metrics-daemon-secret" not found
Jul 17 05:43:55.918624 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:43:55.918590    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9kcb2" podUID="ce41f40b-970f-4698-aa72-e7f772641eaf"
Jul 17 05:43:56.083460 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:43:56.083431    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:43:56.549512 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:43:56.549484    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:44:00.899252 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:44:00.899224    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:44:00.899594 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:44:00.899331    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 05:44:00.899594 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:44:00.899380    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls podName:ce41f40b-970f-4698-aa72-e7f772641eaf nodeName:}" failed. No retries permitted until 2026-07-17 05:46:02.899364931 +0000 UTC m=+285.341303483 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls") pod "dns-default-9kcb2" (UID: "ce41f40b-970f-4698-aa72-e7f772641eaf") : secret "dns-default-metrics-tls" not found
Jul 17 05:44:09.072685 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:44:09.072627    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:44:25.362278 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:44:25.362242    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc" podUID="69a662f9-bf73-47d4-9687-34b7b6b41769" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused"
Jul 17 05:44:25.605887 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:44:25.605867    2558 generic.go:356] "Generic (PLEG): container finished" podID="69a662f9-bf73-47d4-9687-34b7b6b41769" containerID="5e268ef49afe101da0f40f19a859bdf756dfaf9faa7e15530ae4cf6790739129" exitCode=1
Jul 17 05:44:25.605986 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:44:25.605921    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc" event={"ID":"69a662f9-bf73-47d4-9687-34b7b6b41769","Type":"ContainerDied","Data":"5e268ef49afe101da0f40f19a859bdf756dfaf9faa7e15530ae4cf6790739129"}
Jul 17 05:44:25.606169 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:44:25.606158    2558 scope.go:117] "RemoveContainer" containerID="5e268ef49afe101da0f40f19a859bdf756dfaf9faa7e15530ae4cf6790739129"
Jul 17 05:44:26.610838 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:44:26.610814    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc" event={"ID":"69a662f9-bf73-47d4-9687-34b7b6b41769","Type":"ContainerStarted","Data":"e80280a5746a15ee340ea4ee67ee9f25478b258e786ca542a0c5be12ade2b92c"}
Jul 17 05:44:26.611158 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:44:26.611065    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:44:26.611554 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:44:26.611539    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:45:15.475389 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.475365    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"]
Jul 17 05:45:15.477211 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.477193    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76979dcc65-kngkm"]
Jul 17 05:45:15.477331 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.477311    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"
Jul 17 05:45:15.478872 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.478856    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.480218 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.480192    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 17 05:45:15.480295 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.480254    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 17 05:45:15.480537 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.480521    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 17 05:45:15.480748 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.480733    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 17 05:45:15.481351 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.481336    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-28km8\""
Jul 17 05:45:15.481427 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.481341    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hxf9n\""
Jul 17 05:45:15.481427 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.481401    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 17 05:45:15.481590 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.481576    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 17 05:45:15.481769 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.481756    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 17 05:45:15.489236 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.489216    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 17 05:45:15.491459 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.491436    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"]
Jul 17 05:45:15.504716 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.504697    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76979dcc65-kngkm"]
Jul 17 05:45:15.566351 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.566326    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-r6qx5"]
Jul 17 05:45:15.568147 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.568133    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.569562 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.569543    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"]
Jul 17 05:45:15.570602 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.570586    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 17 05:45:15.570685 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.570613    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 17 05:45:15.571270 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.571256    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:15.572294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.572273    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 17 05:45:15.572378 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.572362    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 17 05:45:15.572923 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.572897    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hqbv6\""
Jul 17 05:45:15.573510 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.573497    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 17 05:45:15.573913 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.573823    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 17 05:45:15.574572 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.574556    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4bfj5\""
Jul 17 05:45:15.577517 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.577502    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 17 05:45:15.583840 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.583820    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-r6qx5"]
Jul 17 05:45:15.584490 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.584456    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"]
Jul 17 05:45:15.663285 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.663265    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"]
Jul 17 05:45:15.663747 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.663728    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-image-registry-private-configuration\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.663838 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.663755    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-bound-sa-token\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.663838 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.663786    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-ca-trust-extracted\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.663838 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.663802    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-dnprn\" (UID: \"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"
Jul 17 05:45:15.663934 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.663886    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.663934 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.663907    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-certificates\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.663934 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.663923    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dbgr\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-kube-api-access-2dbgr\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.664022 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.663936    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-dnprn\" (UID: \"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"
Jul 17 05:45:15.664022 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.663963    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-installation-pull-secrets\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.664022 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.663989    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-trusted-ca\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.664022 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.664004    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qfsv\" (UniqueName: \"kubernetes.io/projected/1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7-kube-api-access-5qfsv\") pod \"kube-storage-version-migrator-operator-7658d9b749-dnprn\" (UID: \"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"
Jul 17 05:45:15.664983 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.664971    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"
Jul 17 05:45:15.669314 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.669299    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 17 05:45:15.669415 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.669322    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 17 05:45:15.669747 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.669727    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 17 05:45:15.670418 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.670401    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-sf8lt\""
Jul 17 05:45:15.672163 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.672147    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 17 05:45:15.684009 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.683992    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"]
Jul 17 05:45:15.764377 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764327    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:15.764377 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764361    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-dnprn\" (UID: \"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"
Jul 17 05:45:15.764509 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764380    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqwkf\" (UniqueName: \"kubernetes.io/projected/2a485313-07ed-4e3e-a2b0-30770766d308-kube-api-access-gqwkf\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.764564 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764551    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-certificates\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.764597 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764577    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2a485313-07ed-4e3e-a2b0-30770766d308-serving-cert\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.764625 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764599    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-installation-pull-secrets\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.764625 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764614    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-trusted-ca\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.764764 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764630    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfsv\" (UniqueName: \"kubernetes.io/projected/1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7-kube-api-access-5qfsv\") pod \"kube-storage-version-migrator-operator-7658d9b749-dnprn\" (UID: \"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"
Jul 17 05:45:15.764764 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764655    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-bound-sa-token\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.764764 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764680    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ea296c95-407f-479a-819b-ecf699fb4f83-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:15.764764 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764697    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-image-registry-private-configuration\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.764764 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764714    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2a485313-07ed-4e3e-a2b0-30770766d308-trusted-ca\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.764764 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764754    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-ca-trust-extracted\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.765067 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764782    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dbgr\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-kube-api-access-2dbgr\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.765067 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764807    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a2df3f4e-9b67-4c5c-80df-4770c9101784-serving-cert\") pod \"service-ca-operator-5bcbc94c78-xmhps\" (UID: \"a2df3f4e-9b67-4c5c-80df-4770c9101784\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"
Jul 17 05:45:15.765067 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764830    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2a485313-07ed-4e3e-a2b0-30770766d308-config\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.765067 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764853    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.765067 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764878    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-dnprn\" (UID: \"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"
Jul 17 05:45:15.765067 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764907    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lldkl\" (UniqueName: \"kubernetes.io/projected/a2df3f4e-9b67-4c5c-80df-4770c9101784-kube-api-access-lldkl\") pod \"service-ca-operator-5bcbc94c78-xmhps\" (UID: \"a2df3f4e-9b67-4c5c-80df-4770c9101784\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"
Jul 17 05:45:15.765067 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.764931    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a2df3f4e-9b67-4c5c-80df-4770c9101784-config\") pod \"service-ca-operator-5bcbc94c78-xmhps\" (UID: \"a2df3f4e-9b67-4c5c-80df-4770c9101784\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"
Jul 17 05:45:15.765067 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:15.765068    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 05:45:15.765388 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:15.765082    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76979dcc65-kngkm: secret "image-registry-tls" not found
Jul 17 05:45:15.765388 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.765152    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-ca-trust-extracted\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.765388 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.765224    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-certificates\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.765552 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:15.765499    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls podName:3a48b9a7-2e47-4cce-82f8-1f1e79cd4390 nodeName:}" failed. No retries permitted until 2026-07-17 05:45:16.265445255 +0000 UTC m=+238.707383807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls") pod "image-registry-76979dcc65-kngkm" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390") : secret "image-registry-tls" not found
Jul 17 05:45:15.765791 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.765657    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-dnprn\" (UID: \"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"
Jul 17 05:45:15.765906 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.765886    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-trusted-ca\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.766939 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.766921    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-dnprn\" (UID: \"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"
Jul 17 05:45:15.767053 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.767036    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-installation-pull-secrets\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.767271 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.767255    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-image-registry-private-configuration\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.772842 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.772818    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dbgr\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-kube-api-access-2dbgr\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.772934 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.772861    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qfsv\" (UniqueName: \"kubernetes.io/projected/1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7-kube-api-access-5qfsv\") pod \"kube-storage-version-migrator-operator-7658d9b749-dnprn\" (UID: \"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"
Jul 17 05:45:15.772934 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.772904    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-bound-sa-token\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:15.787764 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.787748    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"
Jul 17 05:45:15.865991 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.865964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:15.866128 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.866003    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqwkf\" (UniqueName: \"kubernetes.io/projected/2a485313-07ed-4e3e-a2b0-30770766d308-kube-api-access-gqwkf\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.866128 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.866040    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2a485313-07ed-4e3e-a2b0-30770766d308-serving-cert\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.866128 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.866096    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ea296c95-407f-479a-819b-ecf699fb4f83-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:15.866128 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.866121    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2a485313-07ed-4e3e-a2b0-30770766d308-trusted-ca\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.866399 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:15.866125    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 05:45:15.866399 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.866168    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a2df3f4e-9b67-4c5c-80df-4770c9101784-serving-cert\") pod \"service-ca-operator-5bcbc94c78-xmhps\" (UID: \"a2df3f4e-9b67-4c5c-80df-4770c9101784\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"
Jul 17 05:45:15.866399 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:15.866194    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert podName:ea296c95-407f-479a-819b-ecf699fb4f83 nodeName:}" failed. No retries permitted until 2026-07-17 05:45:16.366173884 +0000 UTC m=+238.808112436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hx4sr" (UID: "ea296c95-407f-479a-819b-ecf699fb4f83") : secret "networking-console-plugin-cert" not found
Jul 17 05:45:15.866399 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.866230    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2a485313-07ed-4e3e-a2b0-30770766d308-config\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.866399 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.866278    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lldkl\" (UniqueName: \"kubernetes.io/projected/a2df3f4e-9b67-4c5c-80df-4770c9101784-kube-api-access-lldkl\") pod \"service-ca-operator-5bcbc94c78-xmhps\" (UID: \"a2df3f4e-9b67-4c5c-80df-4770c9101784\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"
Jul 17 05:45:15.866399 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.866308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a2df3f4e-9b67-4c5c-80df-4770c9101784-config\") pod \"service-ca-operator-5bcbc94c78-xmhps\" (UID: \"a2df3f4e-9b67-4c5c-80df-4770c9101784\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"
Jul 17 05:45:15.866951 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.866924    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a2df3f4e-9b67-4c5c-80df-4770c9101784-config\") pod \"service-ca-operator-5bcbc94c78-xmhps\" (UID: \"a2df3f4e-9b67-4c5c-80df-4770c9101784\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"
Jul 17 05:45:15.867075 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.867052    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ea296c95-407f-479a-819b-ecf699fb4f83-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:15.867430 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.867410    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2a485313-07ed-4e3e-a2b0-30770766d308-trusted-ca\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.867696 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.867676    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2a485313-07ed-4e3e-a2b0-30770766d308-config\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.868530 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.868510    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a2df3f4e-9b67-4c5c-80df-4770c9101784-serving-cert\") pod \"service-ca-operator-5bcbc94c78-xmhps\" (UID: \"a2df3f4e-9b67-4c5c-80df-4770c9101784\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"
Jul 17 05:45:15.868996 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.868967    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2a485313-07ed-4e3e-a2b0-30770766d308-serving-cert\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.874778 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.874723    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lldkl\" (UniqueName: \"kubernetes.io/projected/a2df3f4e-9b67-4c5c-80df-4770c9101784-kube-api-access-lldkl\") pod \"service-ca-operator-5bcbc94c78-xmhps\" (UID: \"a2df3f4e-9b67-4c5c-80df-4770c9101784\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"
Jul 17 05:45:15.875001 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.874981    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqwkf\" (UniqueName: \"kubernetes.io/projected/2a485313-07ed-4e3e-a2b0-30770766d308-kube-api-access-gqwkf\") pod \"console-operator-565b7b569d-r6qx5\" (UID: \"2a485313-07ed-4e3e-a2b0-30770766d308\") " pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.876690 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.876670    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:15.903636 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.903613    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn"]
Jul 17 05:45:15.906505 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:45:15.906459    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d95d07d_b0e8_4adb_86ef_4ea16b1a21c7.slice/crio-38aa5e3b10e96639f687f19940f84f5c2dc84d23db49065b356d882a98cd501e WatchSource:0}: Error finding container 38aa5e3b10e96639f687f19940f84f5c2dc84d23db49065b356d882a98cd501e: Status 404 returned error can't find the container with id 38aa5e3b10e96639f687f19940f84f5c2dc84d23db49065b356d882a98cd501e
Jul 17 05:45:15.972077 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.972054    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"
Jul 17 05:45:15.990774 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:15.990754    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-r6qx5"]
Jul 17 05:45:15.992947 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:45:15.992911    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a485313_07ed_4e3e_a2b0_30770766d308.slice/crio-2e6fa88056d6c5a236377c5b2735e1d65ddf2b71ce7f85ff998e26a5c569b7d0 WatchSource:0}: Error finding container 2e6fa88056d6c5a236377c5b2735e1d65ddf2b71ce7f85ff998e26a5c569b7d0: Status 404 returned error can't find the container with id 2e6fa88056d6c5a236377c5b2735e1d65ddf2b71ce7f85ff998e26a5c569b7d0
Jul 17 05:45:16.085370 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:16.085346    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps"]
Jul 17 05:45:16.087916 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:45:16.087892    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2df3f4e_9b67_4c5c_80df_4770c9101784.slice/crio-8ef3414f07a01c54b38542b62629e1a22c4b70824af2e1844982c13c543a24e9 WatchSource:0}: Error finding container 8ef3414f07a01c54b38542b62629e1a22c4b70824af2e1844982c13c543a24e9: Status 404 returned error can't find the container with id 8ef3414f07a01c54b38542b62629e1a22c4b70824af2e1844982c13c543a24e9
Jul 17 05:45:16.269140 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:16.269091    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:16.269221 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:16.269214    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 05:45:16.269267 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:16.269224    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76979dcc65-kngkm: secret "image-registry-tls" not found
Jul 17 05:45:16.269267 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:16.269263    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls podName:3a48b9a7-2e47-4cce-82f8-1f1e79cd4390 nodeName:}" failed. No retries permitted until 2026-07-17 05:45:17.26925037 +0000 UTC m=+239.711188922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls") pod "image-registry-76979dcc65-kngkm" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390") : secret "image-registry-tls" not found
Jul 17 05:45:16.369385 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:16.369364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:16.369513 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:16.369499    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 05:45:16.369562 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:16.369547    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert podName:ea296c95-407f-479a-819b-ecf699fb4f83 nodeName:}" failed. No retries permitted until 2026-07-17 05:45:17.369535175 +0000 UTC m=+239.811473726 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hx4sr" (UID: "ea296c95-407f-479a-819b-ecf699fb4f83") : secret "networking-console-plugin-cert" not found
Jul 17 05:45:16.726032 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:16.725983    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps" event={"ID":"a2df3f4e-9b67-4c5c-80df-4770c9101784","Type":"ContainerStarted","Data":"8ef3414f07a01c54b38542b62629e1a22c4b70824af2e1844982c13c543a24e9"}
Jul 17 05:45:16.727617 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:16.727584    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn" event={"ID":"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7","Type":"ContainerStarted","Data":"38aa5e3b10e96639f687f19940f84f5c2dc84d23db49065b356d882a98cd501e"}
Jul 17 05:45:16.729092 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:16.729068    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5" event={"ID":"2a485313-07ed-4e3e-a2b0-30770766d308","Type":"ContainerStarted","Data":"2e6fa88056d6c5a236377c5b2735e1d65ddf2b71ce7f85ff998e26a5c569b7d0"}
Jul 17 05:45:17.276374 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:17.276343    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:17.276610 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:17.276542    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 05:45:17.276610 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:17.276560    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76979dcc65-kngkm: secret "image-registry-tls" not found
Jul 17 05:45:17.276766 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:17.276625    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls podName:3a48b9a7-2e47-4cce-82f8-1f1e79cd4390 nodeName:}" failed. No retries permitted until 2026-07-17 05:45:19.276605554 +0000 UTC m=+241.718544106 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls") pod "image-registry-76979dcc65-kngkm" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390") : secret "image-registry-tls" not found
Jul 17 05:45:17.376943 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:17.376917    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:17.377086 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:17.377058    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 05:45:17.377142 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:17.377124    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert podName:ea296c95-407f-479a-819b-ecf699fb4f83 nodeName:}" failed. No retries permitted until 2026-07-17 05:45:19.377101256 +0000 UTC m=+241.819039814 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hx4sr" (UID: "ea296c95-407f-479a-819b-ecf699fb4f83") : secret "networking-console-plugin-cert" not found
Jul 17 05:45:18.953218 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.953185    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2ktjt"]
Jul 17 05:45:18.956149 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.956123    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:18.959333 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.959309    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 05:45:18.959485 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.959339    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8zw9n\""
Jul 17 05:45:18.959485 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.959397    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 05:45:18.960794 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.960775    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 05:45:18.960794 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.960777    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 05:45:18.967238 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.967217    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2ktjt"]
Jul 17 05:45:18.989000 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.988972    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ad0ac4a-8e85-4e71-9970-cd21695740fb-crio-socket\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:18.989112 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.989032    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ad0ac4a-8e85-4e71-9970-cd21695740fb-data-volume\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:18.989112 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.989076    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlkk4\" (UniqueName: \"kubernetes.io/projected/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-api-access-wlkk4\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:18.989211 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.989115    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:18.989211 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:18.989145    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:19.090342 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.090308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ad0ac4a-8e85-4e71-9970-cd21695740fb-crio-socket\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:19.090453 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.090380    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ad0ac4a-8e85-4e71-9970-cd21695740fb-data-volume\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:19.090453 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.090415    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlkk4\" (UniqueName: \"kubernetes.io/projected/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-api-access-wlkk4\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:19.090453 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.090430    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ad0ac4a-8e85-4e71-9970-cd21695740fb-crio-socket\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:19.090453 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.090449    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:19.090729 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.090495    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:19.090729 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.090592    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:19.090729 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.090628    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 05:45:19.090729 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.090657    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls podName:2ad0ac4a-8e85-4e71-9970-cd21695740fb nodeName:}" failed. No retries permitted until 2026-07-17 05:45:19.590639143 +0000 UTC m=+242.032577708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2ktjt" (UID: "2ad0ac4a-8e85-4e71-9970-cd21695740fb") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:19.090729 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.090684    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm podName:2ad0ac4a-8e85-4e71-9970-cd21695740fb nodeName:}" failed. No retries permitted until 2026-07-17 05:45:19.590667801 +0000 UTC m=+242.032606380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2ktjt" (UID: "2ad0ac4a-8e85-4e71-9970-cd21695740fb") : configmap "kube-rbac-proxy" not found
Jul 17 05:45:19.090729 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.090700    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ad0ac4a-8e85-4e71-9970-cd21695740fb-data-volume\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:19.100523 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.100504    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlkk4\" (UniqueName: \"kubernetes.io/projected/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-api-access-wlkk4\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:19.291803 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.291775    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:19.291948 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.291930    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 05:45:19.292002 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.291950    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76979dcc65-kngkm: secret "image-registry-tls" not found
Jul 17 05:45:19.292049 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.292003    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls podName:3a48b9a7-2e47-4cce-82f8-1f1e79cd4390 nodeName:}" failed. No retries permitted until 2026-07-17 05:45:23.291988092 +0000 UTC m=+245.733926644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls") pod "image-registry-76979dcc65-kngkm" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390") : secret "image-registry-tls" not found
Jul 17 05:45:19.392798 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.392771    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:19.392898 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.392880    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 05:45:19.392957 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.392947    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert podName:ea296c95-407f-479a-819b-ecf699fb4f83 nodeName:}" failed. No retries permitted until 2026-07-17 05:45:23.392930533 +0000 UTC m=+245.834869088 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hx4sr" (UID: "ea296c95-407f-479a-819b-ecf699fb4f83") : secret "networking-console-plugin-cert" not found
Jul 17 05:45:19.594164 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.594139    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:19.594273 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.594187    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:19.594273 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.594260    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:19.594379 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.594323    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 05:45:19.594379 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.594327    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls podName:2ad0ac4a-8e85-4e71-9970-cd21695740fb nodeName:}" failed. No retries permitted until 2026-07-17 05:45:20.594309483 +0000 UTC m=+243.036248056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2ktjt" (UID: "2ad0ac4a-8e85-4e71-9970-cd21695740fb") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:19.594457 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:19.594386    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm podName:2ad0ac4a-8e85-4e71-9970-cd21695740fb nodeName:}" failed. No retries permitted until 2026-07-17 05:45:20.594368927 +0000 UTC m=+243.036307485 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2ktjt" (UID: "2ad0ac4a-8e85-4e71-9970-cd21695740fb") : configmap "kube-rbac-proxy" not found
Jul 17 05:45:19.741609 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.741580    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps" event={"ID":"a2df3f4e-9b67-4c5c-80df-4770c9101784","Type":"ContainerStarted","Data":"0a5c3554eb1a9e97d6308b046202444f34b9808bf035c990edc35933781ad3fa"}
Jul 17 05:45:19.743094 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.743065    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn" event={"ID":"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7","Type":"ContainerStarted","Data":"cac040a5a621245cc166e01588ee868afac393d4031657517e12ac6ba2529de2"}
Jul 17 05:45:19.744679 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.744663    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/0.log"
Jul 17 05:45:19.744773 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.744694    2558 generic.go:356] "Generic (PLEG): container finished" podID="2a485313-07ed-4e3e-a2b0-30770766d308" containerID="d1a500c70566afcaa452a4232e847ecb97e43bfcaaeb078badfe4fa627a07a88" exitCode=255
Jul 17 05:45:19.744773 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.744730    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5" event={"ID":"2a485313-07ed-4e3e-a2b0-30770766d308","Type":"ContainerDied","Data":"d1a500c70566afcaa452a4232e847ecb97e43bfcaaeb078badfe4fa627a07a88"}
Jul 17 05:45:19.744889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.744877    2558 scope.go:117] "RemoveContainer" containerID="d1a500c70566afcaa452a4232e847ecb97e43bfcaaeb078badfe4fa627a07a88"
Jul 17 05:45:19.757312 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.757268    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps" podStartSLOduration=1.7788347949999999 podStartE2EDuration="4.757252034s" totalImagesPullingTime="2.978417239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:45:19.756394622 +0000 UTC m=+242.198333195" watchObservedRunningTime="2026-07-17 05:45:19.757252034 +0000 UTC m=+242.199190608"
Jul 17 05:45:19.787545 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:19.787500    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn" podStartSLOduration=1.631993895 podStartE2EDuration="4.787463786s" totalImagesPullingTime="3.155469891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:45:19.786322922 +0000 UTC m=+242.228261496" watchObservedRunningTime="2026-07-17 05:45:19.787463786 +0000 UTC m=+242.229402362"
Jul 17 05:45:20.601516 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.601463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:20.601830 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.601521    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:20.601830 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:20.601627    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:20.601830 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:20.601633    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 05:45:20.601830 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:20.601675    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls podName:2ad0ac4a-8e85-4e71-9970-cd21695740fb nodeName:}" failed. No retries permitted until 2026-07-17 05:45:22.601662829 +0000 UTC m=+245.043601383 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2ktjt" (UID: "2ad0ac4a-8e85-4e71-9970-cd21695740fb") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:20.601830 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:20.601687    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm podName:2ad0ac4a-8e85-4e71-9970-cd21695740fb nodeName:}" failed. No retries permitted until 2026-07-17 05:45:22.601681424 +0000 UTC m=+245.043619976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2ktjt" (UID: "2ad0ac4a-8e85-4e71-9970-cd21695740fb") : configmap "kube-rbac-proxy" not found
Jul 17 05:45:20.664456 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.664428    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t"]
Jul 17 05:45:20.666296 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.666281    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t"
Jul 17 05:45:20.668786 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.668767    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 17 05:45:20.668786 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.668778    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-x2mwk\""
Jul 17 05:45:20.669989 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.669973    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 17 05:45:20.674825 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.674804    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t"]
Jul 17 05:45:20.702492 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.702451    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv26m\" (UniqueName: \"kubernetes.io/projected/5d81fdd7-e4aa-4faf-a69e-588eac54439c-kube-api-access-fv26m\") pod \"migrator-7cc9c7ffd7-rbl5t\" (UID: \"5d81fdd7-e4aa-4faf-a69e-588eac54439c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t"
Jul 17 05:45:20.750776 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.750761    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 05:45:20.751139 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.751125    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/0.log"
Jul 17 05:45:20.751221 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.751162    2558 generic.go:356] "Generic (PLEG): container finished" podID="2a485313-07ed-4e3e-a2b0-30770766d308" containerID="ac439e1642278752823904382af2d961e1975ed81a50af8e6158bc826836be5c" exitCode=255
Jul 17 05:45:20.751270 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.751239    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5" event={"ID":"2a485313-07ed-4e3e-a2b0-30770766d308","Type":"ContainerDied","Data":"ac439e1642278752823904382af2d961e1975ed81a50af8e6158bc826836be5c"}
Jul 17 05:45:20.751310 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.751297    2558 scope.go:117] "RemoveContainer" containerID="d1a500c70566afcaa452a4232e847ecb97e43bfcaaeb078badfe4fa627a07a88"
Jul 17 05:45:20.751428 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.751414    2558 scope.go:117] "RemoveContainer" containerID="ac439e1642278752823904382af2d961e1975ed81a50af8e6158bc826836be5c"
Jul 17 05:45:20.751642 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:20.751624    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-r6qx5_openshift-console-operator(2a485313-07ed-4e3e-a2b0-30770766d308)\"" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5" podUID="2a485313-07ed-4e3e-a2b0-30770766d308"
Jul 17 05:45:20.803708 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.803683    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv26m\" (UniqueName: \"kubernetes.io/projected/5d81fdd7-e4aa-4faf-a69e-588eac54439c-kube-api-access-fv26m\") pod \"migrator-7cc9c7ffd7-rbl5t\" (UID: \"5d81fdd7-e4aa-4faf-a69e-588eac54439c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t"
Jul 17 05:45:20.811117 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.811096    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv26m\" (UniqueName: \"kubernetes.io/projected/5d81fdd7-e4aa-4faf-a69e-588eac54439c-kube-api-access-fv26m\") pod \"migrator-7cc9c7ffd7-rbl5t\" (UID: \"5d81fdd7-e4aa-4faf-a69e-588eac54439c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t"
Jul 17 05:45:20.975525 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:20.975463    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t"
Jul 17 05:45:21.086307 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:21.086281    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t"]
Jul 17 05:45:21.088869 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:45:21.088843    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d81fdd7_e4aa_4faf_a69e_588eac54439c.slice/crio-ace86f2c868889effd833969fd131e08833cf004dd682b1132eb8855a0d30107 WatchSource:0}: Error finding container ace86f2c868889effd833969fd131e08833cf004dd682b1132eb8855a0d30107: Status 404 returned error can't find the container with id ace86f2c868889effd833969fd131e08833cf004dd682b1132eb8855a0d30107
Jul 17 05:45:21.227597 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:21.227547    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lfzc2_2a31ea27-5883-4df9-9032-72ad9c0ee4ea/dns-node-resolver/0.log"
Jul 17 05:45:21.756431 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:21.756403    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 05:45:21.756806 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:21.756735    2558 scope.go:117] "RemoveContainer" containerID="ac439e1642278752823904382af2d961e1975ed81a50af8e6158bc826836be5c"
Jul 17 05:45:21.756918 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:21.756895    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-r6qx5_openshift-console-operator(2a485313-07ed-4e3e-a2b0-30770766d308)\"" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5" podUID="2a485313-07ed-4e3e-a2b0-30770766d308"
Jul 17 05:45:21.757753 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:21.757723    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t" event={"ID":"5d81fdd7-e4aa-4faf-a69e-588eac54439c","Type":"ContainerStarted","Data":"ace86f2c868889effd833969fd131e08833cf004dd682b1132eb8855a0d30107"}
Jul 17 05:45:21.827240 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:21.827222    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6drsh_19e2915f-ebfc-45ff-9452-9e0c35a105ca/node-ca/0.log"
Jul 17 05:45:22.618064 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:22.618041    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:22.618154 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:22.618073    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:22.618191 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:22.618157    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 05:45:22.618191 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:22.618180    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:22.618248 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:22.618211    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm podName:2ad0ac4a-8e85-4e71-9970-cd21695740fb nodeName:}" failed. No retries permitted until 2026-07-17 05:45:26.618196011 +0000 UTC m=+249.060134565 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2ktjt" (UID: "2ad0ac4a-8e85-4e71-9970-cd21695740fb") : configmap "kube-rbac-proxy" not found
Jul 17 05:45:22.618248 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:22.618224    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls podName:2ad0ac4a-8e85-4e71-9970-cd21695740fb nodeName:}" failed. No retries permitted until 2026-07-17 05:45:26.618218989 +0000 UTC m=+249.060157540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2ktjt" (UID: "2ad0ac4a-8e85-4e71-9970-cd21695740fb") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:22.763356 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:22.762849    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t" event={"ID":"5d81fdd7-e4aa-4faf-a69e-588eac54439c","Type":"ContainerStarted","Data":"d1e8b1a2dbbf61f7e97b6fb5ae8c6902a61ff628ef8c1bea4945673e11e8d85c"}
Jul 17 05:45:22.763356 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:22.762907    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t" event={"ID":"5d81fdd7-e4aa-4faf-a69e-588eac54439c","Type":"ContainerStarted","Data":"0b81bf24d113766e3192af726b367616dfc1f667b1d0b263e549d35250c67893"}
Jul 17 05:45:22.778505 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:22.778446    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbl5t" podStartSLOduration=1.7233357470000001 podStartE2EDuration="2.778434724s" totalImagesPullingTime="1.055098977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:45:22.777897147 +0000 UTC m=+245.219835719" watchObservedRunningTime="2026-07-17 05:45:22.778434724 +0000 UTC m=+245.220373296"
Jul 17 05:45:22.827423 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:22.827404    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dwxmp_480c28d5-a4e5-4bef-9605-a0ceff3cc4b6/serve-healthcheck-canary/0.log"
Jul 17 05:45:23.323513 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:23.323491    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:23.323596 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:23.323568    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 05:45:23.323596 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:23.323585    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76979dcc65-kngkm: secret "image-registry-tls" not found
Jul 17 05:45:23.323664 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:23.323624    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls podName:3a48b9a7-2e47-4cce-82f8-1f1e79cd4390 nodeName:}" failed. No retries permitted until 2026-07-17 05:45:31.323612503 +0000 UTC m=+253.765551055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls") pod "image-registry-76979dcc65-kngkm" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390") : secret "image-registry-tls" not found
Jul 17 05:45:23.424013 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:23.423986    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:23.424096 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:23.424077    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 05:45:23.424136 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:23.424117    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert podName:ea296c95-407f-479a-819b-ecf699fb4f83 nodeName:}" failed. No retries permitted until 2026-07-17 05:45:31.42410619 +0000 UTC m=+253.866044742 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hx4sr" (UID: "ea296c95-407f-479a-819b-ecf699fb4f83") : secret "networking-console-plugin-cert" not found
Jul 17 05:45:25.877023 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:25.876999    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:25.877329 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:25.877032    2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:25.877329 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:25.877280    2558 scope.go:117] "RemoveContainer" containerID="ac439e1642278752823904382af2d961e1975ed81a50af8e6158bc826836be5c"
Jul 17 05:45:25.877421 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:25.877405    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-r6qx5_openshift-console-operator(2a485313-07ed-4e3e-a2b0-30770766d308)\"" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5" podUID="2a485313-07ed-4e3e-a2b0-30770766d308"
Jul 17 05:45:26.644043 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:26.644012    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:26.644043 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:26.644048    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:26.644236 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:26.644125    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 05:45:26.644236 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:26.644148    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:26.644236 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:26.644168    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm podName:2ad0ac4a-8e85-4e71-9970-cd21695740fb nodeName:}" failed. No retries permitted until 2026-07-17 05:45:34.644156835 +0000 UTC m=+257.086095386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2ktjt" (UID: "2ad0ac4a-8e85-4e71-9970-cd21695740fb") : configmap "kube-rbac-proxy" not found
Jul 17 05:45:26.644236 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:26.644199    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls podName:2ad0ac4a-8e85-4e71-9970-cd21695740fb nodeName:}" failed. No retries permitted until 2026-07-17 05:45:34.64418255 +0000 UTC m=+257.086121133 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2ktjt" (UID: "2ad0ac4a-8e85-4e71-9970-cd21695740fb") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:30.071002 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:30.070977    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:45:30.071335 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:30.071065    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 17 05:45:30.071335 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:30.071111    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs podName:e92b27e5-e21d-46ed-8a43-212e4f441cae nodeName:}" failed. No retries permitted until 2026-07-17 05:47:32.071098285 +0000 UTC m=+374.513036838 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs") pod "network-metrics-daemon-tnctb" (UID: "e92b27e5-e21d-46ed-8a43-212e4f441cae") : secret "metrics-daemon-secret" not found
Jul 17 05:45:31.380852 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:31.380827    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:31.382943 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:31.382925    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls\") pod \"image-registry-76979dcc65-kngkm\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") " pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:31.395307 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:31.395289    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hxf9n\""
Jul 17 05:45:31.403336 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:31.403322    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:31.481899 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:31.481873    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:31.482054 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:31.482037    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 05:45:31.482121 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:31.482112    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert podName:ea296c95-407f-479a-819b-ecf699fb4f83 nodeName:}" failed. No retries permitted until 2026-07-17 05:45:47.482094621 +0000 UTC m=+269.924033191 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-hx4sr" (UID: "ea296c95-407f-479a-819b-ecf699fb4f83") : secret "networking-console-plugin-cert" not found
Jul 17 05:45:31.520417 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:31.520387    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76979dcc65-kngkm"]
Jul 17 05:45:31.522935 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:45:31.522908    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a48b9a7_2e47_4cce_82f8_1f1e79cd4390.slice/crio-31c6e69656ca3b0704b3d47c039ba74213ec284b65ad952e12522293e61600bb WatchSource:0}: Error finding container 31c6e69656ca3b0704b3d47c039ba74213ec284b65ad952e12522293e61600bb: Status 404 returned error can't find the container with id 31c6e69656ca3b0704b3d47c039ba74213ec284b65ad952e12522293e61600bb
Jul 17 05:45:31.786633 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:31.786559    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76979dcc65-kngkm" event={"ID":"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390","Type":"ContainerStarted","Data":"a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370"}
Jul 17 05:45:31.786633 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:31.786603    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76979dcc65-kngkm" event={"ID":"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390","Type":"ContainerStarted","Data":"31c6e69656ca3b0704b3d47c039ba74213ec284b65ad952e12522293e61600bb"}
Jul 17 05:45:31.786788 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:31.786660    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:45:31.804708 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:31.804660    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76979dcc65-kngkm" podStartSLOduration=16.804646789 podStartE2EDuration="16.804646789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:45:31.803541741 +0000 UTC m=+254.245480315" watchObservedRunningTime="2026-07-17 05:45:31.804646789 +0000 UTC m=+254.246585361"
Jul 17 05:45:34.705379 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:34.705344    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:34.705816 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:34.705387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:34.705816 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:34.705461    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:34.705816 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:34.705532    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls podName:2ad0ac4a-8e85-4e71-9970-cd21695740fb nodeName:}" failed. No retries permitted until 2026-07-17 05:45:50.705516396 +0000 UTC m=+273.147454950 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2ktjt" (UID: "2ad0ac4a-8e85-4e71-9970-cd21695740fb") : secret "insights-runtime-extractor-tls" not found
Jul 17 05:45:34.705996 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:34.705963    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad0ac4a-8e85-4e71-9970-cd21695740fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:41.072110 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.072045    2558 scope.go:117] "RemoveContainer" containerID="ac439e1642278752823904382af2d961e1975ed81a50af8e6158bc826836be5c"
Jul 17 05:45:41.598762 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.598733    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76979dcc65-kngkm"]
Jul 17 05:45:41.636086 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.636061    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76dcbff9f5-lfp28"]
Jul 17 05:45:41.642149 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.642131    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.650270 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.650250    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76dcbff9f5-lfp28"]
Jul 17 05:45:41.755952 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.755921    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8887bdcd-8c10-401a-b2cb-bef7ac87108d-bound-sa-token\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.756193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.756173    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl58h\" (UniqueName: \"kubernetes.io/projected/8887bdcd-8c10-401a-b2cb-bef7ac87108d-kube-api-access-jl58h\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.756588 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.756562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8887bdcd-8c10-401a-b2cb-bef7ac87108d-trusted-ca\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.756734 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.756717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8887bdcd-8c10-401a-b2cb-bef7ac87108d-installation-pull-secrets\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.756896 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.756873    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8887bdcd-8c10-401a-b2cb-bef7ac87108d-ca-trust-extracted\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.757011 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.756920    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8887bdcd-8c10-401a-b2cb-bef7ac87108d-registry-certificates\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.757011 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.756960    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8887bdcd-8c10-401a-b2cb-bef7ac87108d-image-registry-private-configuration\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.757124 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.757009    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8887bdcd-8c10-401a-b2cb-bef7ac87108d-registry-tls\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.816573 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.816549    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 05:45:41.816705 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.816607    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5" event={"ID":"2a485313-07ed-4e3e-a2b0-30770766d308","Type":"ContainerStarted","Data":"f1a71f8576dc772b4bdc572a48c3c36e6473c51af082c65aa2c9dc19a522f9b9"}
Jul 17 05:45:41.816903 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.816869    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:41.821063 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.821045    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5"
Jul 17 05:45:41.836278 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.836231    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-r6qx5" podStartSLOduration=23.761370185 podStartE2EDuration="26.836216759s" totalImagesPullingTime="3.074846574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:45:41.835499174 +0000 UTC m=+264.277437757" watchObservedRunningTime="2026-07-17 05:45:41.836216759 +0000 UTC m=+264.278155332"
Jul 17 05:45:41.857888 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.857820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8887bdcd-8c10-401a-b2cb-bef7ac87108d-image-registry-private-configuration\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.857888 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.857873    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8887bdcd-8c10-401a-b2cb-bef7ac87108d-registry-tls\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.858054 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.857949    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8887bdcd-8c10-401a-b2cb-bef7ac87108d-bound-sa-token\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.858054 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.857985    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl58h\" (UniqueName: \"kubernetes.io/projected/8887bdcd-8c10-401a-b2cb-bef7ac87108d-kube-api-access-jl58h\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.858054 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.858038    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8887bdcd-8c10-401a-b2cb-bef7ac87108d-trusted-ca\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.858198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.858081    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8887bdcd-8c10-401a-b2cb-bef7ac87108d-installation-pull-secrets\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.858198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.858134    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8887bdcd-8c10-401a-b2cb-bef7ac87108d-ca-trust-extracted\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.858198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.858181    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8887bdcd-8c10-401a-b2cb-bef7ac87108d-registry-certificates\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.859076 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.859055    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8887bdcd-8c10-401a-b2cb-bef7ac87108d-registry-certificates\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.859510 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.859341    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8887bdcd-8c10-401a-b2cb-bef7ac87108d-ca-trust-extracted\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.860415 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.860262    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8887bdcd-8c10-401a-b2cb-bef7ac87108d-trusted-ca\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.861983 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.861935    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8887bdcd-8c10-401a-b2cb-bef7ac87108d-installation-pull-secrets\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.861983 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.861952    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8887bdcd-8c10-401a-b2cb-bef7ac87108d-registry-tls\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.862726 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.862702    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8887bdcd-8c10-401a-b2cb-bef7ac87108d-image-registry-private-configuration\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.866986 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.866965    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8887bdcd-8c10-401a-b2cb-bef7ac87108d-bound-sa-token\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.867797 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.867778    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl58h\" (UniqueName: \"kubernetes.io/projected/8887bdcd-8c10-401a-b2cb-bef7ac87108d-kube-api-access-jl58h\") pod \"image-registry-76dcbff9f5-lfp28\" (UID: \"8887bdcd-8c10-401a-b2cb-bef7ac87108d\") " pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:41.950620 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:41.950594    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:42.106188 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:42.106163    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76dcbff9f5-lfp28"]
Jul 17 05:45:42.109222 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:45:42.109168    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8887bdcd_8c10_401a_b2cb_bef7ac87108d.slice/crio-1f8b0281a2aba2339df159554cfb6ae7450fc08fb0c0f6dd1b2c7a842c08190f WatchSource:0}: Error finding container 1f8b0281a2aba2339df159554cfb6ae7450fc08fb0c0f6dd1b2c7a842c08190f: Status 404 returned error can't find the container with id 1f8b0281a2aba2339df159554cfb6ae7450fc08fb0c0f6dd1b2c7a842c08190f
Jul 17 05:45:42.820908 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:42.820876    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" event={"ID":"8887bdcd-8c10-401a-b2cb-bef7ac87108d","Type":"ContainerStarted","Data":"983a250e1ba92c68007535521e286a06c1f6e0199babb00d191c2dc067415aa3"}
Jul 17 05:45:42.824979 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:42.821407    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" event={"ID":"8887bdcd-8c10-401a-b2cb-bef7ac87108d","Type":"ContainerStarted","Data":"1f8b0281a2aba2339df159554cfb6ae7450fc08fb0c0f6dd1b2c7a842c08190f"}
Jul 17 05:45:42.842172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:42.842124    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" podStartSLOduration=1.842108218 podStartE2EDuration="1.842108218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:45:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:45:42.840948256 +0000 UTC m=+265.282886843" watchObservedRunningTime="2026-07-17 05:45:42.842108218 +0000 UTC m=+265.284046792"
Jul 17 05:45:43.823742 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:43.823715    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:45:47.498784 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:47.498756    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:47.500843 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:47.500816    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ea296c95-407f-479a-819b-ecf699fb4f83-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-hx4sr\" (UID: \"ea296c95-407f-479a-819b-ecf699fb4f83\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:47.683609 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:47.683587    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4bfj5\""
Jul 17 05:45:47.691726 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:47.691709    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"
Jul 17 05:45:47.808859 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:47.808834    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr"]
Jul 17 05:45:47.812056 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:45:47.812027    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea296c95_407f_479a_819b_ecf699fb4f83.slice/crio-2ce9c428d7e39af3211542a6b1e67c8b05cc9cba5d331834c9e80167ebb69e55 WatchSource:0}: Error finding container 2ce9c428d7e39af3211542a6b1e67c8b05cc9cba5d331834c9e80167ebb69e55: Status 404 returned error can't find the container with id 2ce9c428d7e39af3211542a6b1e67c8b05cc9cba5d331834c9e80167ebb69e55
Jul 17 05:45:47.835847 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:47.835822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr" event={"ID":"ea296c95-407f-479a-819b-ecf699fb4f83","Type":"ContainerStarted","Data":"2ce9c428d7e39af3211542a6b1e67c8b05cc9cba5d331834c9e80167ebb69e55"}
Jul 17 05:45:49.842538 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:49.842503    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr" event={"ID":"ea296c95-407f-479a-819b-ecf699fb4f83","Type":"ContainerStarted","Data":"34fa41aabc4b1ee14b31570303c0d3b65ca9fafdfab6477cd4911241fa3326ce"}
Jul 17 05:45:49.857801 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:49.857751    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-hx4sr" podStartSLOduration=33.703697292 podStartE2EDuration="34.857738143s" totalImagesPullingTime="1.154040851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:45:49.857267026 +0000 UTC m=+272.299205617" watchObservedRunningTime="2026-07-17 05:45:49.857738143 +0000 UTC m=+272.299676775"
Jul 17 05:45:50.557879 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.557851    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2"]
Jul 17 05:45:50.560056 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.560043    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2"
Jul 17 05:45:50.562511 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.562495    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 17 05:45:50.562688 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.562668    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lsvkm\""
Jul 17 05:45:50.567227 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.567205    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2"]
Jul 17 05:45:50.620092 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.620072    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f878a236-a96b-4020-85fa-b5faa8813f00-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-zjgh2\" (UID: \"f878a236-a96b-4020-85fa-b5faa8813f00\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2"
Jul 17 05:45:50.720373 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.720356    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:50.720467 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.720390    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f878a236-a96b-4020-85fa-b5faa8813f00-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-zjgh2\" (UID: \"f878a236-a96b-4020-85fa-b5faa8813f00\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2"
Jul 17 05:45:50.722623 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.722602    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad0ac4a-8e85-4e71-9970-cd21695740fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ktjt\" (UID: \"2ad0ac4a-8e85-4e71-9970-cd21695740fb\") " pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:50.722745 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.722726    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f878a236-a96b-4020-85fa-b5faa8813f00-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-zjgh2\" (UID: \"f878a236-a96b-4020-85fa-b5faa8813f00\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2"
Jul 17 05:45:50.767053 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.767033    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2ktjt"
Jul 17 05:45:50.868978 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.868931    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2"
Jul 17 05:45:50.884401 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.884379    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2ktjt"]
Jul 17 05:45:50.998080 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:50.998052    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2"]
Jul 17 05:45:51.000817 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:45:51.000794    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf878a236_a96b_4020_85fa_b5faa8813f00.slice/crio-596e5026f7ac3da1fa99cf02a63942ce55f648d97f12b2337d480f56615c04e4 WatchSource:0}: Error finding container 596e5026f7ac3da1fa99cf02a63942ce55f648d97f12b2337d480f56615c04e4: Status 404 returned error can't find the container with id 596e5026f7ac3da1fa99cf02a63942ce55f648d97f12b2337d480f56615c04e4
Jul 17 05:45:51.605885 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:51.605830    2558 patch_prober.go:28] interesting pod/image-registry-76979dcc65-kngkm container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 17 05:45:51.606051 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:51.605916    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76979dcc65-kngkm" podUID="3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:45:51.850013 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:51.849977    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2" event={"ID":"f878a236-a96b-4020-85fa-b5faa8813f00","Type":"ContainerStarted","Data":"596e5026f7ac3da1fa99cf02a63942ce55f648d97f12b2337d480f56615c04e4"}
Jul 17 05:45:51.851249 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:51.851218    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ktjt" event={"ID":"2ad0ac4a-8e85-4e71-9970-cd21695740fb","Type":"ContainerStarted","Data":"8b2814d71e9a87ebee194e861ffa1149cc31686784a6e9d090cd5bdc572e3f84"}
Jul 17 05:45:52.856659 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:52.856628    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2" event={"ID":"f878a236-a96b-4020-85fa-b5faa8813f00","Type":"ContainerStarted","Data":"c2f1b7bb58edf7cf4758d6a1ecdd867df8f74a16adba10f4ff52a863834dc463"}
Jul 17 05:45:52.857144 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:52.856846    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2"
Jul 17 05:45:52.861848 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:52.861825    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2"
Jul 17 05:45:52.872045 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:52.872002    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zjgh2" podStartSLOduration=1.7455292839999998 podStartE2EDuration="2.87198734s" totalImagesPullingTime="1.126458056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:45:52.871656169 +0000 UTC m=+275.313594741" watchObservedRunningTime="2026-07-17 05:45:52.87198734 +0000 UTC m=+275.313925915"
Jul 17 05:45:55.870117 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:55.870054    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ktjt" event={"ID":"2ad0ac4a-8e85-4e71-9970-cd21695740fb","Type":"ContainerStarted","Data":"d4703ffea728270b79d52ce2ae1926a16db3e81d654bbc83570ae7f975f8b466"}
Jul 17 05:45:59.550670 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:45:59.550635    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9kcb2" podUID="ce41f40b-970f-4698-aa72-e7f772641eaf"
Jul 17 05:45:59.884785 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:59.884743    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ktjt" event={"ID":"2ad0ac4a-8e85-4e71-9970-cd21695740fb","Type":"ContainerStarted","Data":"29d211beb5678411c63f2df26eb2db09342e4dbe04033ae9eda08afcaccfa654"}
Jul 17 05:45:59.884937 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:45:59.884836    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:46:01.604313 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:01.604276    2558 patch_prober.go:28] interesting pod/image-registry-76979dcc65-kngkm container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 17 05:46:01.604862 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:01.604365    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76979dcc65-kngkm" podUID="3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:46:01.956082 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:01.955997    2558 patch_prober.go:28] interesting pod/image-registry-76dcbff9f5-lfp28 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 17 05:46:01.956082 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:01.956058    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" podUID="8887bdcd-8c10-401a-b2cb-bef7ac87108d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:46:02.895259 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:02.895234    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ktjt" event={"ID":"2ad0ac4a-8e85-4e71-9970-cd21695740fb","Type":"ContainerStarted","Data":"fcfa6c71b27c8852a1741b8576dbe0f53243f6a5848d1bc1bbfe27d28202775b"}
Jul 17 05:46:02.913002 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:02.912958    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2ktjt" podStartSLOduration=33.584873943 podStartE2EDuration="44.912945447s" totalImagesPullingTime="11.328071504s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:45:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:46:02.911379852 +0000 UTC m=+285.353318451" watchObservedRunningTime="2026-07-17 05:46:02.912945447 +0000 UTC m=+285.354884019"
Jul 17 05:46:02.917819 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:02.917800    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:46:02.919823 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:02.919799    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce41f40b-970f-4698-aa72-e7f772641eaf-metrics-tls\") pod \"dns-default-9kcb2\" (UID: \"ce41f40b-970f-4698-aa72-e7f772641eaf\") " pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:46:03.188149 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:03.188106    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jx7lr\""
Jul 17 05:46:03.196305 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:03.196289    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:46:03.312092 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:03.312069    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9kcb2"]
Jul 17 05:46:03.314358 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:46:03.314330    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce41f40b_970f_4698_aa72_e7f772641eaf.slice/crio-df542e6e75fc91cf5725f5e03c0fe04a96a98b09af928fd1f2f298f323be71a3 WatchSource:0}: Error finding container df542e6e75fc91cf5725f5e03c0fe04a96a98b09af928fd1f2f298f323be71a3: Status 404 returned error can't find the container with id df542e6e75fc91cf5725f5e03c0fe04a96a98b09af928fd1f2f298f323be71a3
Jul 17 05:46:03.900307 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:03.900275    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9kcb2" event={"ID":"ce41f40b-970f-4698-aa72-e7f772641eaf","Type":"ContainerStarted","Data":"df542e6e75fc91cf5725f5e03c0fe04a96a98b09af928fd1f2f298f323be71a3"}
Jul 17 05:46:04.003925 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.003901    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9wndm"]
Jul 17 05:46:04.007255 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.007234    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.009730 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.009707    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 05:46:04.009963 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.009930    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 05:46:04.010102 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.010083    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 05:46:04.011190 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.011126    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 05:46:04.011281 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.011129    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 05:46:04.011281 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.011246    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dzjkz\""
Jul 17 05:46:04.011389 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.011132    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 05:46:04.126160 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.126138    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97d15fd8-7e65-4d18-b11d-56854f7d380b-sys\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.126366 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.126204    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-wtmp\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.126366 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.126232    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97d15fd8-7e65-4d18-b11d-56854f7d380b-metrics-client-ca\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.126366 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.126257    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-textfile\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.126366 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.126276    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-tls\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.126366 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.126300    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97d15fd8-7e65-4d18-b11d-56854f7d380b-root\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.126366 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.126323    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjnsc\" (UniqueName: \"kubernetes.io/projected/97d15fd8-7e65-4d18-b11d-56854f7d380b-kube-api-access-jjnsc\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.126668 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.126360    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.126668 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.126395    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-accelerators-collector-config\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.226972 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.226909    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-wtmp\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.226972 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.226948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97d15fd8-7e65-4d18-b11d-56854f7d380b-metrics-client-ca\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227138 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.226982    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-textfile\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227138 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227011    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-tls\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227138 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227038    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97d15fd8-7e65-4d18-b11d-56854f7d380b-root\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227138 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227062    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjnsc\" (UniqueName: \"kubernetes.io/projected/97d15fd8-7e65-4d18-b11d-56854f7d380b-kube-api-access-jjnsc\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227138 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227090    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-wtmp\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227138 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227104    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227138 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227130    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97d15fd8-7e65-4d18-b11d-56854f7d380b-root\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227457 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:46:04.227151    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 17 05:46:04.227457 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:46:04.227215    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-tls podName:97d15fd8-7e65-4d18-b11d-56854f7d380b nodeName:}" failed. No retries permitted until 2026-07-17 05:46:04.727194493 +0000 UTC m=+287.169133075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-tls") pod "node-exporter-9wndm" (UID: "97d15fd8-7e65-4d18-b11d-56854f7d380b") : secret "node-exporter-tls" not found
Jul 17 05:46:04.227457 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227134    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-accelerators-collector-config\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227457 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227328    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97d15fd8-7e65-4d18-b11d-56854f7d380b-sys\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227457 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227452    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97d15fd8-7e65-4d18-b11d-56854f7d380b-sys\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227737 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227516    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-textfile\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227737 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227648    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97d15fd8-7e65-4d18-b11d-56854f7d380b-metrics-client-ca\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.227836 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.227736    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-accelerators-collector-config\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.229707 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.229678    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.240008 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.239991    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjnsc\" (UniqueName: \"kubernetes.io/projected/97d15fd8-7e65-4d18-b11d-56854f7d380b-kube-api-access-jjnsc\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.732343 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.732304    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-tls\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.735085 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.735055    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97d15fd8-7e65-4d18-b11d-56854f7d380b-node-exporter-tls\") pod \"node-exporter-9wndm\" (UID: \"97d15fd8-7e65-4d18-b11d-56854f7d380b\") " pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.831930 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.831902    2558 patch_prober.go:28] interesting pod/image-registry-76dcbff9f5-lfp28 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 17 05:46:04.832154 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.832123    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" podUID="8887bdcd-8c10-401a-b2cb-bef7ac87108d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:46:04.906594 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.906530    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9kcb2" event={"ID":"ce41f40b-970f-4698-aa72-e7f772641eaf","Type":"ContainerStarted","Data":"151e019db36fa23e0f5bfce3b96b8ef4a5bd2bf4d640168759c180187a37b1ee"}
Jul 17 05:46:04.919773 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:04.919204    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9wndm"
Jul 17 05:46:04.933455 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:46:04.933432    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97d15fd8_7e65_4d18_b11d_56854f7d380b.slice/crio-958f674610768e9013fba457e8e668ed6559a601710277cee5bba44318ac72b7 WatchSource:0}: Error finding container 958f674610768e9013fba457e8e668ed6559a601710277cee5bba44318ac72b7: Status 404 returned error can't find the container with id 958f674610768e9013fba457e8e668ed6559a601710277cee5bba44318ac72b7
Jul 17 05:46:05.912222 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:05.912194    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9kcb2" event={"ID":"ce41f40b-970f-4698-aa72-e7f772641eaf","Type":"ContainerStarted","Data":"f4893b3e9050c5805769d900044f55e5a1a4e0f9c43e0fdbb81ed5638676c549"}
Jul 17 05:46:05.912540 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:05.912333    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:46:05.913391 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:05.913365    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9wndm" event={"ID":"97d15fd8-7e65-4d18-b11d-56854f7d380b","Type":"ContainerStarted","Data":"958f674610768e9013fba457e8e668ed6559a601710277cee5bba44318ac72b7"}
Jul 17 05:46:05.931439 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:05.931343    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9kcb2" podStartSLOduration=252.446511895 podStartE2EDuration="4m13.931330811s" totalImagesPullingTime="1.484818916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:46:05.929624539 +0000 UTC m=+288.371563148" watchObservedRunningTime="2026-07-17 05:46:05.931330811 +0000 UTC m=+288.373269381"
Jul 17 05:46:06.617693 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.617659    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76979dcc65-kngkm" podUID="3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" containerName="registry" containerID="cri-o://a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370" gracePeriod=30
Jul 17 05:46:06.855655 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.855633    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:46:06.917644 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.917577    2558 generic.go:356] "Generic (PLEG): container finished" podID="3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" containerID="a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370" exitCode=0
Jul 17 05:46:06.917644 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.917616    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76979dcc65-kngkm"
Jul 17 05:46:06.918007 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.917671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76979dcc65-kngkm" event={"ID":"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390","Type":"ContainerDied","Data":"a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370"}
Jul 17 05:46:06.918007 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.917720    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76979dcc65-kngkm" event={"ID":"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390","Type":"ContainerDied","Data":"31c6e69656ca3b0704b3d47c039ba74213ec284b65ad952e12522293e61600bb"}
Jul 17 05:46:06.918007 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.917745    2558 scope.go:117] "RemoveContainer" containerID="a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370"
Jul 17 05:46:06.919389 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.919370    2558 generic.go:356] "Generic (PLEG): container finished" podID="97d15fd8-7e65-4d18-b11d-56854f7d380b" containerID="ae34221782005de81598e3d528de372558090bb6403797e31d3b47f04843d038" exitCode=0
Jul 17 05:46:06.919508 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.919450    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9wndm" event={"ID":"97d15fd8-7e65-4d18-b11d-56854f7d380b","Type":"ContainerDied","Data":"ae34221782005de81598e3d528de372558090bb6403797e31d3b47f04843d038"}
Jul 17 05:46:06.925986 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.925963    2558 scope.go:117] "RemoveContainer" containerID="a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370"
Jul 17 05:46:06.926263 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:46:06.926244    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370\": container with ID starting with a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370 not found: ID does not exist" containerID="a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370"
Jul 17 05:46:06.926315 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.926273    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370"} err="failed to get container status \"a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370\": rpc error: code = NotFound desc = could not find container \"a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370\": container with ID starting with a38cd70c14fbe41be2fc5459b0feb1ee36813fcb7c78781a919e0c70c348b370 not found: ID does not exist"
Jul 17 05:46:06.950139 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.950117    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dbgr\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-kube-api-access-2dbgr\") pod \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") "
Jul 17 05:46:06.950224 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.950154    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-installation-pull-secrets\") pod \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") "
Jul 17 05:46:06.950224 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.950188    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-ca-trust-extracted\") pod \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") "
Jul 17 05:46:06.950224 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.950209    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-trusted-ca\") pod \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") "
Jul 17 05:46:06.950363 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.950227    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls\") pod \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") "
Jul 17 05:46:06.950363 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.950247    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-certificates\") pod \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") "
Jul 17 05:46:06.950654 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.950628    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:46:06.950750 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.950649    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:46:06.950812 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.950752    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-image-registry-private-configuration\") pod \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") "
Jul 17 05:46:06.950812 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.950795    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-bound-sa-token\") pod \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\" (UID: \"3a48b9a7-2e47-4cce-82f8-1f1e79cd4390\") "
Jul 17 05:46:06.951007 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.950991    2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-certificates\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:46:06.951071 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.951011    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-trusted-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:46:06.952379 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.952352    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-kube-api-access-2dbgr" (OuterVolumeSpecName: "kube-api-access-2dbgr") pod "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390"). InnerVolumeSpecName "kube-api-access-2dbgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:46:06.952530 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.952484    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:46:06.952782 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.952757    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:46:06.952866 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.952803    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:46:06.953042 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.953017    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:46:06.958512 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.958489    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" (UID: "3a48b9a7-2e47-4cce-82f8-1f1e79cd4390"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:46:06.976909 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.976887    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"]
Jul 17 05:46:06.977128 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.977116    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" containerName="registry"
Jul 17 05:46:06.977165 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.977130    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" containerName="registry"
Jul 17 05:46:06.977198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.977177    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" containerName="registry"
Jul 17 05:46:06.982107 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.982089    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:06.984604 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.984590    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 17 05:46:06.984604 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.984596    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 17 05:46:06.984846 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.984830    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 17 05:46:06.984918 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.984888    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hks87\""
Jul 17 05:46:06.984976 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.984930    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 17 05:46:06.984976 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.984938    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 17 05:46:06.984976 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.984954    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3c65ibq3rtu97\""
Jul 17 05:46:06.991445 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:06.991379    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"]
Jul 17 05:46:07.051582 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.051360    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dbgr\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-kube-api-access-2dbgr\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:46:07.051582 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.051386    2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-installation-pull-secrets\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:46:07.051582 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.051400    2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-ca-trust-extracted\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:46:07.051582 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.051429    2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-registry-tls\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:46:07.051582 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.051442    2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-image-registry-private-configuration\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:46:07.051582 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.051457    2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390-bound-sa-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:46:07.152403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.152377    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.152534 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.152409    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.152534 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.152437    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.152534 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.152524    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.152637 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.152545    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21c5ac76-68fa-4aaa-9125-bcf4944538f9-metrics-client-ca\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.152637 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.152588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-grpc-tls\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.152637 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.152623    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-tls\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.152721 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.152643    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x56j6\" (UniqueName: \"kubernetes.io/projected/21c5ac76-68fa-4aaa-9125-bcf4944538f9-kube-api-access-x56j6\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.237681 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.237658    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76979dcc65-kngkm"]
Jul 17 05:46:07.241031 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.241011    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76979dcc65-kngkm"]
Jul 17 05:46:07.253451 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.253429    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-tls\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.253553 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.253463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x56j6\" (UniqueName: \"kubernetes.io/projected/21c5ac76-68fa-4aaa-9125-bcf4944538f9-kube-api-access-x56j6\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.253553 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.253520    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.253553 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.253548    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.253689 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.253594    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.253689 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.253649    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.253689 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.253676    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21c5ac76-68fa-4aaa-9125-bcf4944538f9-metrics-client-ca\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.253803 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.253707    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-grpc-tls\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.254583 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.254555    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21c5ac76-68fa-4aaa-9125-bcf4944538f9-metrics-client-ca\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.256261 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.256217    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.256377 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.256356    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-tls\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.256629 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.256609    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.256788 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.256764    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.256857 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.256844    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-grpc-tls\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.257087 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.257065    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21c5ac76-68fa-4aaa-9125-bcf4944538f9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.261268 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.261247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x56j6\" (UniqueName: \"kubernetes.io/projected/21c5ac76-68fa-4aaa-9125-bcf4944538f9-kube-api-access-x56j6\") pod \"thanos-querier-7c959ff9bd-tm2sq\" (UID: \"21c5ac76-68fa-4aaa-9125-bcf4944538f9\") " pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.290223 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.290200    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:07.411251 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.411223    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"]
Jul 17 05:46:07.414320 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:46:07.414286    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21c5ac76_68fa_4aaa_9125_bcf4944538f9.slice/crio-6e96e9b92fbeb58487c67f0ffd5b7151504964aff731ded0762546d875266830 WatchSource:0}: Error finding container 6e96e9b92fbeb58487c67f0ffd5b7151504964aff731ded0762546d875266830: Status 404 returned error can't find the container with id 6e96e9b92fbeb58487c67f0ffd5b7151504964aff731ded0762546d875266830
Jul 17 05:46:07.924714 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.924686    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9wndm" event={"ID":"97d15fd8-7e65-4d18-b11d-56854f7d380b","Type":"ContainerStarted","Data":"3c1521e6f7e0b8329943740cdc810a86e69ea6a472faae61572271c83d64f298"}
Jul 17 05:46:07.925066 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.924751    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9wndm" event={"ID":"97d15fd8-7e65-4d18-b11d-56854f7d380b","Type":"ContainerStarted","Data":"c1d2d484ca6bfed4bb544ac750eecac697457453e0726819c67699228b09ae26"}
Jul 17 05:46:07.926350 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.926328    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq" event={"ID":"21c5ac76-68fa-4aaa-9125-bcf4944538f9","Type":"ContainerStarted","Data":"6e96e9b92fbeb58487c67f0ffd5b7151504964aff731ded0762546d875266830"}
Jul 17 05:46:07.943849 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:07.943812    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9wndm" podStartSLOduration=3.941478827 podStartE2EDuration="4.943798327s" totalImagesPullingTime="936.438505ms" totalInitContainerRuntime="65.880995ms" isStatefulPod=true podCreationTimestamp="2026-07-17 05:46:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:46:07.942369244 +0000 UTC m=+290.384307841" watchObservedRunningTime="2026-07-17 05:46:07.943798327 +0000 UTC m=+290.385736901"
Jul 17 05:46:08.076933 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.076910    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a48b9a7-2e47-4cce-82f8-1f1e79cd4390" path="/var/lib/kubelet/pods/3a48b9a7-2e47-4cce-82f8-1f1e79cd4390/volumes"
Jul 17 05:46:08.394529 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.394465    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6c48c9cf85-78hkc"]
Jul 17 05:46:08.403930 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.403905    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.407308 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.407283    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c48c9cf85-78hkc"]
Jul 17 05:46:08.407563 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.407541    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 17 05:46:08.407667 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.407542    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 17 05:46:08.407723 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.407667    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 17 05:46:08.407723 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.407685    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c66gckij5s4t4\""
Jul 17 05:46:08.407812 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.407723    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 17 05:46:08.408102 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.408084    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5nfx2\""
Jul 17 05:46:08.562274 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.562245    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/08563a71-8da4-4270-9370-e3ad8aaa4c9d-metrics-server-audit-profiles\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.562394 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.562287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/08563a71-8da4-4270-9370-e3ad8aaa4c9d-secret-metrics-server-client-certs\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.562394 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.562303    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/08563a71-8da4-4270-9370-e3ad8aaa4c9d-secret-metrics-server-tls\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.562394 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.562327    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08563a71-8da4-4270-9370-e3ad8aaa4c9d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.562394 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.562361    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/08563a71-8da4-4270-9370-e3ad8aaa4c9d-client-ca-bundle\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.562599 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.562411    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxr9r\" (UniqueName: \"kubernetes.io/projected/08563a71-8da4-4270-9370-e3ad8aaa4c9d-kube-api-access-nxr9r\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.562599 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.562456    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/08563a71-8da4-4270-9370-e3ad8aaa4c9d-audit-log\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.663590 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.663527    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/08563a71-8da4-4270-9370-e3ad8aaa4c9d-metrics-server-audit-profiles\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.663590 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.663577    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/08563a71-8da4-4270-9370-e3ad8aaa4c9d-secret-metrics-server-client-certs\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.663751 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.663594    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/08563a71-8da4-4270-9370-e3ad8aaa4c9d-secret-metrics-server-tls\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.663751 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.663619    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08563a71-8da4-4270-9370-e3ad8aaa4c9d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.663751 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.663640    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/08563a71-8da4-4270-9370-e3ad8aaa4c9d-client-ca-bundle\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.663751 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.663680    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxr9r\" (UniqueName: \"kubernetes.io/projected/08563a71-8da4-4270-9370-e3ad8aaa4c9d-kube-api-access-nxr9r\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.663751 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.663742    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/08563a71-8da4-4270-9370-e3ad8aaa4c9d-audit-log\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.664523 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.664336    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/08563a71-8da4-4270-9370-e3ad8aaa4c9d-audit-log\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.664617 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.664588    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08563a71-8da4-4270-9370-e3ad8aaa4c9d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.664832 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.664634    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/08563a71-8da4-4270-9370-e3ad8aaa4c9d-metrics-server-audit-profiles\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.666610 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.666583    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/08563a71-8da4-4270-9370-e3ad8aaa4c9d-client-ca-bundle\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.666723 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.666700    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/08563a71-8da4-4270-9370-e3ad8aaa4c9d-secret-metrics-server-tls\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.666811 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.666757    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/08563a71-8da4-4270-9370-e3ad8aaa4c9d-secret-metrics-server-client-certs\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.671376 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.671356    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxr9r\" (UniqueName: \"kubernetes.io/projected/08563a71-8da4-4270-9370-e3ad8aaa4c9d-kube-api-access-nxr9r\") pod \"metrics-server-6c48c9cf85-78hkc\" (UID: \"08563a71-8da4-4270-9370-e3ad8aaa4c9d\") " pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.714339 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.714306    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:08.774321 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.774299    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"]
Jul 17 05:46:08.778910 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.778215    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"
Jul 17 05:46:08.781019 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.780995    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 17 05:46:08.781190 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.780995    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-7p7jz\""
Jul 17 05:46:08.787889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.787864    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"]
Jul 17 05:46:08.852943 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.852919    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c48c9cf85-78hkc"]
Jul 17 05:46:08.856422 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:46:08.856270    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08563a71_8da4_4270_9370_e3ad8aaa4c9d.slice/crio-b31f91de1c70cb51785a7fa1ebbfe638a696252099bece17e5f565c9d2ca3ae3 WatchSource:0}: Error finding container b31f91de1c70cb51785a7fa1ebbfe638a696252099bece17e5f565c9d2ca3ae3: Status 404 returned error can't find the container with id b31f91de1c70cb51785a7fa1ebbfe638a696252099bece17e5f565c9d2ca3ae3
Jul 17 05:46:08.866076 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.866050    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cfcg\" (UniqueName: \"kubernetes.io/projected/76e15cb7-000d-4462-aceb-2cc1a38b23ab-kube-api-access-4cfcg\") pod \"monitoring-plugin-7b7b7bc8bf-5zxhh\" (UID: \"76e15cb7-000d-4462-aceb-2cc1a38b23ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"
Jul 17 05:46:08.866166 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.866106    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76e15cb7-000d-4462-aceb-2cc1a38b23ab-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-5zxhh\" (UID: \"76e15cb7-000d-4462-aceb-2cc1a38b23ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"
Jul 17 05:46:08.931571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.931517    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc" event={"ID":"08563a71-8da4-4270-9370-e3ad8aaa4c9d","Type":"ContainerStarted","Data":"b31f91de1c70cb51785a7fa1ebbfe638a696252099bece17e5f565c9d2ca3ae3"}
Jul 17 05:46:08.967096 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.967072    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cfcg\" (UniqueName: \"kubernetes.io/projected/76e15cb7-000d-4462-aceb-2cc1a38b23ab-kube-api-access-4cfcg\") pod \"monitoring-plugin-7b7b7bc8bf-5zxhh\" (UID: \"76e15cb7-000d-4462-aceb-2cc1a38b23ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"
Jul 17 05:46:08.967186 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.967106    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76e15cb7-000d-4462-aceb-2cc1a38b23ab-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-5zxhh\" (UID: \"76e15cb7-000d-4462-aceb-2cc1a38b23ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"
Jul 17 05:46:08.967234 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:46:08.967205    2558 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 17 05:46:08.967268 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:46:08.967249    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76e15cb7-000d-4462-aceb-2cc1a38b23ab-monitoring-plugin-cert podName:76e15cb7-000d-4462-aceb-2cc1a38b23ab nodeName:}" failed. No retries permitted until 2026-07-17 05:46:09.467236066 +0000 UTC m=+291.909174618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/76e15cb7-000d-4462-aceb-2cc1a38b23ab-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-5zxhh" (UID: "76e15cb7-000d-4462-aceb-2cc1a38b23ab") : secret "monitoring-plugin-cert" not found
Jul 17 05:46:08.975251 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:08.975230    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cfcg\" (UniqueName: \"kubernetes.io/projected/76e15cb7-000d-4462-aceb-2cc1a38b23ab-kube-api-access-4cfcg\") pod \"monitoring-plugin-7b7b7bc8bf-5zxhh\" (UID: \"76e15cb7-000d-4462-aceb-2cc1a38b23ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"
Jul 17 05:46:09.472168 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:09.472132    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76e15cb7-000d-4462-aceb-2cc1a38b23ab-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-5zxhh\" (UID: \"76e15cb7-000d-4462-aceb-2cc1a38b23ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"
Jul 17 05:46:09.474969 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:09.474944    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76e15cb7-000d-4462-aceb-2cc1a38b23ab-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-5zxhh\" (UID: \"76e15cb7-000d-4462-aceb-2cc1a38b23ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"
Jul 17 05:46:09.688331 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:09.688294    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"
Jul 17 05:46:09.817618 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:09.817592    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"]
Jul 17 05:46:09.826487 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:46:09.826447    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76e15cb7_000d_4462_aceb_2cc1a38b23ab.slice/crio-94dad4fe83ef7a93cf64b29bda43cb92a6cec8de66e445b14d9e3b75db5ea766 WatchSource:0}: Error finding container 94dad4fe83ef7a93cf64b29bda43cb92a6cec8de66e445b14d9e3b75db5ea766: Status 404 returned error can't find the container with id 94dad4fe83ef7a93cf64b29bda43cb92a6cec8de66e445b14d9e3b75db5ea766
Jul 17 05:46:09.937889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:09.937714    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq" event={"ID":"21c5ac76-68fa-4aaa-9125-bcf4944538f9","Type":"ContainerStarted","Data":"974977e6e6e29ca69929d67bd46d6fe366c15b963b22810cc1119196abbd7d06"}
Jul 17 05:46:09.937889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:09.937799    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq" event={"ID":"21c5ac76-68fa-4aaa-9125-bcf4944538f9","Type":"ContainerStarted","Data":"58c41f09a300dbfc53d36bba9d69af3d63280880e11a01b5f945115727f5ab72"}
Jul 17 05:46:09.937889 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:09.937843    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq" event={"ID":"21c5ac76-68fa-4aaa-9125-bcf4944538f9","Type":"ContainerStarted","Data":"e33f160fd50590cad6cb69ee872af027db355e216b80edecf70af1c5747d3ab1"}
Jul 17 05:46:09.939724 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:09.939666    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh" event={"ID":"76e15cb7-000d-4462-aceb-2cc1a38b23ab","Type":"ContainerStarted","Data":"94dad4fe83ef7a93cf64b29bda43cb92a6cec8de66e445b14d9e3b75db5ea766"}
Jul 17 05:46:11.948654 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.948618    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc" event={"ID":"08563a71-8da4-4270-9370-e3ad8aaa4c9d","Type":"ContainerStarted","Data":"71dc4db87c104925a4218902f01be522510e4398a39b0ca4c1f6c4dd0450e9c2"}
Jul 17 05:46:11.952027 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.952002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq" event={"ID":"21c5ac76-68fa-4aaa-9125-bcf4944538f9","Type":"ContainerStarted","Data":"69420f5e39a690324228775d0d38110b7b6beee6b39c14087829ebbe11735890"}
Jul 17 05:46:11.952151 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.952069    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq" event={"ID":"21c5ac76-68fa-4aaa-9125-bcf4944538f9","Type":"ContainerStarted","Data":"b40b434e5c1349d2930571ebfa1c746e7fef1aa0d71b00eb2d3f7b5ef2d2e15d"}
Jul 17 05:46:11.952151 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.952114    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:11.952151 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.952148    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq" event={"ID":"21c5ac76-68fa-4aaa-9125-bcf4944538f9","Type":"ContainerStarted","Data":"9630e983dcef29cee5e8f5d474a53378c6d53632bffedce560b57140ee168a63"}
Jul 17 05:46:11.953563 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.953537    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh" event={"ID":"76e15cb7-000d-4462-aceb-2cc1a38b23ab","Type":"ContainerStarted","Data":"63d72c66c6af70a428ff9ae527e325874e89d2f66bfd0fcfaad86309badd31de"}
Jul 17 05:46:11.953756 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.953736    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"
Jul 17 05:46:11.955003 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.954983    2558 patch_prober.go:28] interesting pod/image-registry-76dcbff9f5-lfp28 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 17 05:46:11.955096 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.955020    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" podUID="8887bdcd-8c10-401a-b2cb-bef7ac87108d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:46:11.957931 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.957901    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh"
Jul 17 05:46:11.964131 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.964095    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc" podStartSLOduration=1.474820866 podStartE2EDuration="3.964084258s" totalImagesPullingTime="2.489263392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:46:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:46:11.963789828 +0000 UTC m=+294.405728401" watchObservedRunningTime="2026-07-17 05:46:11.964084258 +0000 UTC m=+294.406022875"
Jul 17 05:46:11.978042 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.978006    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5zxhh" podStartSLOduration=2.455453477 podStartE2EDuration="3.97799477s" totalImagesPullingTime="1.522541293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:46:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:46:11.977341632 +0000 UTC m=+294.419280221" watchObservedRunningTime="2026-07-17 05:46:11.97799477 +0000 UTC m=+294.419933399"
Jul 17 05:46:11.999284 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:11.999251    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq" podStartSLOduration=2.247179806 podStartE2EDuration="5.999236132s" totalImagesPullingTime="3.752056326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:46:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:46:11.997747051 +0000 UTC m=+294.439685623" watchObservedRunningTime="2026-07-17 05:46:11.999236132 +0000 UTC m=+294.441174709"
Jul 17 05:46:12.073662 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:46:12.073634    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-tnctb" podUID="e92b27e5-e21d-46ed-8a43-212e4f441cae"
Jul 17 05:46:14.830948 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:14.830916    2558 patch_prober.go:28] interesting pod/image-registry-76dcbff9f5-lfp28 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 17 05:46:14.831270 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:14.830968    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" podUID="8887bdcd-8c10-401a-b2cb-bef7ac87108d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:46:15.921749 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:15.921721    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9kcb2"
Jul 17 05:46:17.915425 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:17.915399    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 05:46:17.915997 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:17.915974    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 05:46:17.921169 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:17.921150    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 05:46:17.921350 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:17.921334    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 05:46:17.926967 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:17.926949    2558 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 05:46:17.962631 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:17.962610    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7c959ff9bd-tm2sq"
Jul 17 05:46:21.954827 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:21.954797    2558 patch_prober.go:28] interesting pod/image-registry-76dcbff9f5-lfp28 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 17 05:46:21.960546 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:21.954858    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" podUID="8887bdcd-8c10-401a-b2cb-bef7ac87108d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:46:21.960546 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:21.954902    2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:46:21.960546 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:21.955328    2558 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"983a250e1ba92c68007535521e286a06c1f6e0199babb00d191c2dc067415aa3"} pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 17 05:46:21.960546 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:21.958752    2558 patch_prober.go:28] interesting pod/image-registry-76dcbff9f5-lfp28 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 17 05:46:21.960546 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:21.958786    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" podUID="8887bdcd-8c10-401a-b2cb-bef7ac87108d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:46:24.076005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:24.075983    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:46:24.995780 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:24.995732    2558 generic.go:356] "Generic (PLEG): container finished" podID="a2df3f4e-9b67-4c5c-80df-4770c9101784" containerID="0a5c3554eb1a9e97d6308b046202444f34b9808bf035c990edc35933781ad3fa" exitCode=0
Jul 17 05:46:24.995863 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:24.995776    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps" event={"ID":"a2df3f4e-9b67-4c5c-80df-4770c9101784","Type":"ContainerDied","Data":"0a5c3554eb1a9e97d6308b046202444f34b9808bf035c990edc35933781ad3fa"}
Jul 17 05:46:24.996015 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:24.996002    2558 scope.go:117] "RemoveContainer" containerID="0a5c3554eb1a9e97d6308b046202444f34b9808bf035c990edc35933781ad3fa"
Jul 17 05:46:26.001055 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:26.001004    2558 generic.go:356] "Generic (PLEG): container finished" podID="69a662f9-bf73-47d4-9687-34b7b6b41769" containerID="e80280a5746a15ee340ea4ee67ee9f25478b258e786ca542a0c5be12ade2b92c" exitCode=1
Jul 17 05:46:26.001397 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:26.001083    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc" event={"ID":"69a662f9-bf73-47d4-9687-34b7b6b41769","Type":"ContainerDied","Data":"e80280a5746a15ee340ea4ee67ee9f25478b258e786ca542a0c5be12ade2b92c"}
Jul 17 05:46:26.001397 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:26.001168    2558 scope.go:117] "RemoveContainer" containerID="5e268ef49afe101da0f40f19a859bdf756dfaf9faa7e15530ae4cf6790739129"
Jul 17 05:46:26.001515 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:26.001456    2558 scope.go:117] "RemoveContainer" containerID="e80280a5746a15ee340ea4ee67ee9f25478b258e786ca542a0c5be12ade2b92c"
Jul 17 05:46:26.001671 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:46:26.001646    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-d54864cb-pjqwc_open-cluster-management-agent-addon(69a662f9-bf73-47d4-9687-34b7b6b41769)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc" podUID="69a662f9-bf73-47d4-9687-34b7b6b41769"
Jul 17 05:46:26.002959 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:26.002939    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xmhps" event={"ID":"a2df3f4e-9b67-4c5c-80df-4770c9101784","Type":"ContainerStarted","Data":"f043ef892bc410d6b038a81504d5e77144657fb5004c9a32ace215d7de3ebf70"}
Jul 17 05:46:26.611758 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:26.611733    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:46:27.007780 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:27.007717    2558 scope.go:117] "RemoveContainer" containerID="e80280a5746a15ee340ea4ee67ee9f25478b258e786ca542a0c5be12ade2b92c"
Jul 17 05:46:27.008067 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:46:27.007869    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-d54864cb-pjqwc_open-cluster-management-agent-addon(69a662f9-bf73-47d4-9687-34b7b6b41769)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc" podUID="69a662f9-bf73-47d4-9687-34b7b6b41769"
Jul 17 05:46:28.715347 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:28.715317    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:28.715699 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:28.715359    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:30.711393 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:30.711367    2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:46:30.711750 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:30.711651    2558 scope.go:117] "RemoveContainer" containerID="e80280a5746a15ee340ea4ee67ee9f25478b258e786ca542a0c5be12ade2b92c"
Jul 17 05:46:30.711791 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:46:30.711767    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-d54864cb-pjqwc_open-cluster-management-agent-addon(69a662f9-bf73-47d4-9687-34b7b6b41769)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc" podUID="69a662f9-bf73-47d4-9687-34b7b6b41769"
Jul 17 05:46:31.959460 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:31.959436    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:46:35.035289 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:35.035223    2558 generic.go:356] "Generic (PLEG): container finished" podID="1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7" containerID="cac040a5a621245cc166e01588ee868afac393d4031657517e12ac6ba2529de2" exitCode=0
Jul 17 05:46:35.035626 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:35.035294    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn" event={"ID":"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7","Type":"ContainerDied","Data":"cac040a5a621245cc166e01588ee868afac393d4031657517e12ac6ba2529de2"}
Jul 17 05:46:35.035626 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:35.035565    2558 scope.go:117] "RemoveContainer" containerID="cac040a5a621245cc166e01588ee868afac393d4031657517e12ac6ba2529de2"
Jul 17 05:46:36.041200 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:36.041168    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-dnprn" event={"ID":"1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7","Type":"ContainerStarted","Data":"e4089dccf5b6d787cfebf7f4905859422c56570a219036bc539b237028b4ab4b"}
Jul 17 05:46:43.073084 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:43.073056    2558 scope.go:117] "RemoveContainer" containerID="e80280a5746a15ee340ea4ee67ee9f25478b258e786ca542a0c5be12ade2b92c"
Jul 17 05:46:44.069603 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:44.069571    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc" event={"ID":"69a662f9-bf73-47d4-9687-34b7b6b41769","Type":"ContainerStarted","Data":"ace09739aeec8ffb2bc976269f28126563ce1fd1a842146d514992fe9c39acb2"}
Jul 17 05:46:44.069862 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:44.069844    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:46:44.071092 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:44.071072    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d54864cb-pjqwc"
Jul 17 05:46:46.973789 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:46.973755    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" podUID="8887bdcd-8c10-401a-b2cb-bef7ac87108d" containerName="registry" containerID="cri-o://983a250e1ba92c68007535521e286a06c1f6e0199babb00d191c2dc067415aa3" gracePeriod=30
Jul 17 05:46:48.719885 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:48.719855    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:48.723608 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:48.723586    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6c48c9cf85-78hkc"
Jul 17 05:46:49.089091 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:49.089064    2558 generic.go:356] "Generic (PLEG): container finished" podID="8887bdcd-8c10-401a-b2cb-bef7ac87108d" containerID="983a250e1ba92c68007535521e286a06c1f6e0199babb00d191c2dc067415aa3" exitCode=0
Jul 17 05:46:49.089218 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:49.089153    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" event={"ID":"8887bdcd-8c10-401a-b2cb-bef7ac87108d","Type":"ContainerDied","Data":"983a250e1ba92c68007535521e286a06c1f6e0199babb00d191c2dc067415aa3"}
Jul 17 05:46:49.089218 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:49.089198    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28" event={"ID":"8887bdcd-8c10-401a-b2cb-bef7ac87108d","Type":"ContainerStarted","Data":"6cb402aa8be9637fb1f03a72634df467908e49131f3a78dafbe74b64c7de4faf"}
Jul 17 05:46:49.089354 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:49.089340    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:46:55.345524 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:55.345501    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-76dcbff9f5-lfp28_8887bdcd-8c10-401a-b2cb-bef7ac87108d/registry/0.log"
Jul 17 05:46:55.351324 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:55.351307    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-76dcbff9f5-lfp28_8887bdcd-8c10-401a-b2cb-bef7ac87108d/registry/1.log"
Jul 17 05:46:55.632878 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:55.632812    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6drsh_19e2915f-ebfc-45ff-9452-9e0c35a105ca/node-ca/0.log"
Jul 17 05:46:56.632913 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:46:56.632895    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dwxmp_480c28d5-a4e5-4bef-9605-a0ceff3cc4b6/serve-healthcheck-canary/0.log"
Jul 17 05:47:10.096435 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:47:10.096368    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76dcbff9f5-lfp28"
Jul 17 05:47:32.140542 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:47:32.140512    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:47:32.142629 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:47:32.142612    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e92b27e5-e21d-46ed-8a43-212e4f441cae-metrics-certs\") pod \"network-metrics-daemon-tnctb\" (UID: \"e92b27e5-e21d-46ed-8a43-212e4f441cae\") " pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:47:32.179725 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:47:32.179701    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7x5wz\""
Jul 17 05:47:32.187779 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:47:32.187759    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnctb"
Jul 17 05:47:32.310697 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:47:32.310677    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tnctb"]
Jul 17 05:47:32.312769 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:47:32.312742    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode92b27e5_e21d_46ed_8a43_212e4f441cae.slice/crio-9861c5fe08f09ace81cb9475170187aed1da4942945273bea755d8f60353a08d WatchSource:0}: Error finding container 9861c5fe08f09ace81cb9475170187aed1da4942945273bea755d8f60353a08d: Status 404 returned error can't find the container with id 9861c5fe08f09ace81cb9475170187aed1da4942945273bea755d8f60353a08d
Jul 17 05:47:32.314671 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:47:32.314651    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:47:33.231237 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:47:33.231190    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tnctb" event={"ID":"e92b27e5-e21d-46ed-8a43-212e4f441cae","Type":"ContainerStarted","Data":"9861c5fe08f09ace81cb9475170187aed1da4942945273bea755d8f60353a08d"}
Jul 17 05:47:34.236806 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:47:34.236777    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tnctb" event={"ID":"e92b27e5-e21d-46ed-8a43-212e4f441cae","Type":"ContainerStarted","Data":"d280b59b8f6a6f93855f533d6471faee578a3cc4d0e995f4043b6a103e87f709"}
Jul 17 05:47:34.237184 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:47:34.236824    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tnctb" event={"ID":"e92b27e5-e21d-46ed-8a43-212e4f441cae","Type":"ContainerStarted","Data":"e07a5f0009c4b126d809027f560b11f5687f9595e0d7e1328660d2cbfb7da2a5"}
Jul 17 05:47:34.252499 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:47:34.252427    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tnctb" podStartSLOduration=375.287547201 podStartE2EDuration="6m16.252408874s" totalImagesPullingTime="964.861673ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:47:34.251635703 +0000 UTC m=+376.693574276" watchObservedRunningTime="2026-07-17 05:47:34.252408874 +0000 UTC m=+376.694347446"
Jul 17 05:49:48.617555 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.617522    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d9644"]
Jul 17 05:49:48.620676 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.620651    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-d9644"
Jul 17 05:49:48.623174 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.623153    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 17 05:49:48.624464 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.624448    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 17 05:49:48.624464 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.624456    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7gwxj\""
Jul 17 05:49:48.629837 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.629818    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d9644"]
Jul 17 05:49:48.668207 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.668182    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wllm7\" (UniqueName: \"kubernetes.io/projected/43a1a5ea-016a-44e0-bb7c-44cec1d3414e-kube-api-access-wllm7\") pod \"cert-manager-webhook-6486ffd997-d9644\" (UID: \"43a1a5ea-016a-44e0-bb7c-44cec1d3414e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9644"
Jul 17 05:49:48.668292 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.668220    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43a1a5ea-016a-44e0-bb7c-44cec1d3414e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d9644\" (UID: \"43a1a5ea-016a-44e0-bb7c-44cec1d3414e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9644"
Jul 17 05:49:48.768917 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.768895    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wllm7\" (UniqueName: \"kubernetes.io/projected/43a1a5ea-016a-44e0-bb7c-44cec1d3414e-kube-api-access-wllm7\") pod \"cert-manager-webhook-6486ffd997-d9644\" (UID: \"43a1a5ea-016a-44e0-bb7c-44cec1d3414e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9644"
Jul 17 05:49:48.769012 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.768929    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43a1a5ea-016a-44e0-bb7c-44cec1d3414e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d9644\" (UID: \"43a1a5ea-016a-44e0-bb7c-44cec1d3414e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9644"
Jul 17 05:49:48.776498 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.776454    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wllm7\" (UniqueName: \"kubernetes.io/projected/43a1a5ea-016a-44e0-bb7c-44cec1d3414e-kube-api-access-wllm7\") pod \"cert-manager-webhook-6486ffd997-d9644\" (UID: \"43a1a5ea-016a-44e0-bb7c-44cec1d3414e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9644"
Jul 17 05:49:48.776583 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.776524    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43a1a5ea-016a-44e0-bb7c-44cec1d3414e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d9644\" (UID: \"43a1a5ea-016a-44e0-bb7c-44cec1d3414e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9644"
Jul 17 05:49:48.943752 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:48.943696    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-d9644"
Jul 17 05:49:49.066196 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:49.066167    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d9644"]
Jul 17 05:49:49.069402 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:49:49.069372    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43a1a5ea_016a_44e0_bb7c_44cec1d3414e.slice/crio-74f9555f6d078949e6bd6bc3cec5d279571682ada1d7d79b1b41b0c6959f595a WatchSource:0}: Error finding container 74f9555f6d078949e6bd6bc3cec5d279571682ada1d7d79b1b41b0c6959f595a: Status 404 returned error can't find the container with id 74f9555f6d078949e6bd6bc3cec5d279571682ada1d7d79b1b41b0c6959f595a
Jul 17 05:49:49.703877 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:49.703839    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-d9644" event={"ID":"43a1a5ea-016a-44e0-bb7c-44cec1d3414e","Type":"ContainerStarted","Data":"74f9555f6d078949e6bd6bc3cec5d279571682ada1d7d79b1b41b0c6959f595a"}
Jul 17 05:49:52.718467 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:52.718424    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-d9644" event={"ID":"43a1a5ea-016a-44e0-bb7c-44cec1d3414e","Type":"ContainerStarted","Data":"f4ad5969f12f3cb5f5e89dbff24101cc4e648ccfd7b23f8f89c39b1695facff2"}
Jul 17 05:49:52.718880 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:52.718561    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-d9644"
Jul 17 05:49:52.734134 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:52.734090    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-d9644" podStartSLOduration=1.771313856 podStartE2EDuration="4.73407899s" totalImagesPullingTime="2.962765134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:49:52.733626526 +0000 UTC m=+515.175565098" watchObservedRunningTime="2026-07-17 05:49:52.73407899 +0000 UTC m=+515.176017562"
Jul 17 05:49:58.723939 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:49:58.723904    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-d9644"
Jul 17 05:50:25.002162 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.002084    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"]
Jul 17 05:50:25.006657 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.006634    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.009609 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.009585    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 05:50:25.009735 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.009586    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 05:50:25.009735 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.009622    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xnd5q\""
Jul 17 05:50:25.009902 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.009887    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 05:50:25.009936 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.009909    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 05:50:25.020792 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.020769    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"]
Jul 17 05:50:25.119890 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.119859    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksc72\" (UniqueName: \"kubernetes.io/projected/7a21be7f-68cb-4458-b0c6-9c2658c01da8-kube-api-access-ksc72\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.120002 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.119912    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7a21be7f-68cb-4458-b0c6-9c2658c01da8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.120002 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.119934    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7a21be7f-68cb-4458-b0c6-9c2658c01da8-manifests\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.120002 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.119992    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a21be7f-68cb-4458-b0c6-9c2658c01da8-webhook-cert\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.120099 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.120020    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a21be7f-68cb-4458-b0c6-9c2658c01da8-tmp\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.220427 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.220400    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksc72\" (UniqueName: \"kubernetes.io/projected/7a21be7f-68cb-4458-b0c6-9c2658c01da8-kube-api-access-ksc72\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.220581 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.220456    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7a21be7f-68cb-4458-b0c6-9c2658c01da8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.220581 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.220508    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7a21be7f-68cb-4458-b0c6-9c2658c01da8-manifests\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.220581 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.220544    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a21be7f-68cb-4458-b0c6-9c2658c01da8-webhook-cert\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.220581 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.220570    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a21be7f-68cb-4458-b0c6-9c2658c01da8-tmp\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.220922 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.220898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7a21be7f-68cb-4458-b0c6-9c2658c01da8-manifests\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.222845 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.222821    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a21be7f-68cb-4458-b0c6-9c2658c01da8-tmp\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.223045 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.223028    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a21be7f-68cb-4458-b0c6-9c2658c01da8-webhook-cert\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.223088 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.223056    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7a21be7f-68cb-4458-b0c6-9c2658c01da8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.229089 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.229070    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksc72\" (UniqueName: \"kubernetes.io/projected/7a21be7f-68cb-4458-b0c6-9c2658c01da8-kube-api-access-ksc72\") pod \"opendatahub-operator-controller-manager-554df894d5-zcxct\" (UID: \"7a21be7f-68cb-4458-b0c6-9c2658c01da8\") " pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.316839 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.316788    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:25.449625 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.449573    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"]
Jul 17 05:50:25.452010 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:50:25.451978    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a21be7f_68cb_4458_b0c6_9c2658c01da8.slice/crio-39e01cd3d26731002d9b45516d0aba392570a464c0c3ff0157a8165d34508395 WatchSource:0}: Error finding container 39e01cd3d26731002d9b45516d0aba392570a464c0c3ff0157a8165d34508395: Status 404 returned error can't find the container with id 39e01cd3d26731002d9b45516d0aba392570a464c0c3ff0157a8165d34508395
Jul 17 05:50:25.837157 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:25.837123    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct" event={"ID":"7a21be7f-68cb-4458-b0c6-9c2658c01da8","Type":"ContainerStarted","Data":"39e01cd3d26731002d9b45516d0aba392570a464c0c3ff0157a8165d34508395"}
Jul 17 05:50:28.857510 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:28.857454    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct" event={"ID":"7a21be7f-68cb-4458-b0c6-9c2658c01da8","Type":"ContainerStarted","Data":"4a0e678801140707ef8b1fb2548a516fd09120abc6b6ad764bd8e4e84a2d22a0"}
Jul 17 05:50:29.862679 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:29.862646    2558 generic.go:356] "Generic (PLEG): container finished" podID="7a21be7f-68cb-4458-b0c6-9c2658c01da8" containerID="4a0e678801140707ef8b1fb2548a516fd09120abc6b6ad764bd8e4e84a2d22a0" exitCode=0
Jul 17 05:50:29.863100 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:29.862731    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct" event={"ID":"7a21be7f-68cb-4458-b0c6-9c2658c01da8","Type":"ContainerDied","Data":"4a0e678801140707ef8b1fb2548a516fd09120abc6b6ad764bd8e4e84a2d22a0"}
Jul 17 05:50:30.868824 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:30.868785    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct" event={"ID":"7a21be7f-68cb-4458-b0c6-9c2658c01da8","Type":"ContainerStarted","Data":"985e5e294fe492254a8c9719fd6e48598dba5af97f958902316f9ff9cf7a9556"}
Jul 17 05:50:30.869202 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:30.868859    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:30.890165 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:30.890111    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct" podStartSLOduration=2.573171996 podStartE2EDuration="6.890095856s" totalImagesPullingTime="3.293089002s" totalInitContainerRuntime="1.023834858s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:50:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:50:30.889199507 +0000 UTC m=+553.331138099" watchObservedRunningTime="2026-07-17 05:50:30.890095856 +0000 UTC m=+553.332034504"
Jul 17 05:50:41.875157 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:41.875128    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-554df894d5-zcxct"
Jul 17 05:50:44.756094 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:44.756061    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jts89"]
Jul 17 05:50:44.761743 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:44.761724    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jts89"
Jul 17 05:50:44.764148 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:44.764131    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 17 05:50:44.764214 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:44.764154    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-whjbt\""
Jul 17 05:50:44.776123 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:44.776100    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jts89"]
Jul 17 05:50:44.865632 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:44.865599    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1ade4bd-5a9c-4a04-b311-ff623b15885d-cert\") pod \"odh-model-controller-7998c67dc4-jts89\" (UID: \"a1ade4bd-5a9c-4a04-b311-ff623b15885d\") " pod="opendatahub/odh-model-controller-7998c67dc4-jts89"
Jul 17 05:50:44.865793 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:44.865683    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh5bl\" (UniqueName: \"kubernetes.io/projected/a1ade4bd-5a9c-4a04-b311-ff623b15885d-kube-api-access-zh5bl\") pod \"odh-model-controller-7998c67dc4-jts89\" (UID: \"a1ade4bd-5a9c-4a04-b311-ff623b15885d\") " pod="opendatahub/odh-model-controller-7998c67dc4-jts89"
Jul 17 05:50:44.966316 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:44.966285    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh5bl\" (UniqueName: \"kubernetes.io/projected/a1ade4bd-5a9c-4a04-b311-ff623b15885d-kube-api-access-zh5bl\") pod \"odh-model-controller-7998c67dc4-jts89\" (UID: \"a1ade4bd-5a9c-4a04-b311-ff623b15885d\") " pod="opendatahub/odh-model-controller-7998c67dc4-jts89"
Jul 17 05:50:44.966504 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:44.966330    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1ade4bd-5a9c-4a04-b311-ff623b15885d-cert\") pod \"odh-model-controller-7998c67dc4-jts89\" (UID: \"a1ade4bd-5a9c-4a04-b311-ff623b15885d\") " pod="opendatahub/odh-model-controller-7998c67dc4-jts89"
Jul 17 05:50:44.966504 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:50:44.966449    2558 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 17 05:50:44.966619 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:50:44.966527    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1ade4bd-5a9c-4a04-b311-ff623b15885d-cert podName:a1ade4bd-5a9c-4a04-b311-ff623b15885d nodeName:}" failed. No retries permitted until 2026-07-17 05:50:45.466510504 +0000 UTC m=+567.908449057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a1ade4bd-5a9c-4a04-b311-ff623b15885d-cert") pod "odh-model-controller-7998c67dc4-jts89" (UID: "a1ade4bd-5a9c-4a04-b311-ff623b15885d") : secret "odh-model-controller-webhook-cert" not found
Jul 17 05:50:44.976942 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:44.976922    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh5bl\" (UniqueName: \"kubernetes.io/projected/a1ade4bd-5a9c-4a04-b311-ff623b15885d-kube-api-access-zh5bl\") pod \"odh-model-controller-7998c67dc4-jts89\" (UID: \"a1ade4bd-5a9c-4a04-b311-ff623b15885d\") " pod="opendatahub/odh-model-controller-7998c67dc4-jts89"
Jul 17 05:50:45.332385 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.332349    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-2tdtj"]
Jul 17 05:50:45.335237 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.335222    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj"
Jul 17 05:50:45.337664 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.337643    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 17 05:50:45.343943 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.343921    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-2tdtj"]
Jul 17 05:50:45.470123 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.470081    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83a2d61b-6c06-4136-916c-01a82ff92723-tls-certs\") pod \"model-serving-api-54d89bd5fc-2tdtj\" (UID: \"83a2d61b-6c06-4136-916c-01a82ff92723\") " pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj"
Jul 17 05:50:45.470292 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.470139    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1ade4bd-5a9c-4a04-b311-ff623b15885d-cert\") pod \"odh-model-controller-7998c67dc4-jts89\" (UID: \"a1ade4bd-5a9c-4a04-b311-ff623b15885d\") " pod="opendatahub/odh-model-controller-7998c67dc4-jts89"
Jul 17 05:50:45.470292 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.470158    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96bjk\" (UniqueName: \"kubernetes.io/projected/83a2d61b-6c06-4136-916c-01a82ff92723-kube-api-access-96bjk\") pod \"model-serving-api-54d89bd5fc-2tdtj\" (UID: \"83a2d61b-6c06-4136-916c-01a82ff92723\") " pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj"
Jul 17 05:50:45.472678 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.472647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1ade4bd-5a9c-4a04-b311-ff623b15885d-cert\") pod \"odh-model-controller-7998c67dc4-jts89\" (UID: \"a1ade4bd-5a9c-4a04-b311-ff623b15885d\") " pod="opendatahub/odh-model-controller-7998c67dc4-jts89"
Jul 17 05:50:45.571490 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.571435    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83a2d61b-6c06-4136-916c-01a82ff92723-tls-certs\") pod \"model-serving-api-54d89bd5fc-2tdtj\" (UID: \"83a2d61b-6c06-4136-916c-01a82ff92723\") " pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj"
Jul 17 05:50:45.571667 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.571521    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96bjk\" (UniqueName: \"kubernetes.io/projected/83a2d61b-6c06-4136-916c-01a82ff92723-kube-api-access-96bjk\") pod \"model-serving-api-54d89bd5fc-2tdtj\" (UID: \"83a2d61b-6c06-4136-916c-01a82ff92723\") " pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj"
Jul 17 05:50:45.571667 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:50:45.571590    2558 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 17 05:50:45.571756 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:50:45.571679    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83a2d61b-6c06-4136-916c-01a82ff92723-tls-certs podName:83a2d61b-6c06-4136-916c-01a82ff92723 nodeName:}" failed. No retries permitted until 2026-07-17 05:50:46.071662059 +0000 UTC m=+568.513600611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/83a2d61b-6c06-4136-916c-01a82ff92723-tls-certs") pod "model-serving-api-54d89bd5fc-2tdtj" (UID: "83a2d61b-6c06-4136-916c-01a82ff92723") : secret "model-serving-api-tls" not found
Jul 17 05:50:45.582195 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.582169    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96bjk\" (UniqueName: \"kubernetes.io/projected/83a2d61b-6c06-4136-916c-01a82ff92723-kube-api-access-96bjk\") pod \"model-serving-api-54d89bd5fc-2tdtj\" (UID: \"83a2d61b-6c06-4136-916c-01a82ff92723\") " pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj"
Jul 17 05:50:45.671409 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.671323    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jts89"
Jul 17 05:50:45.794898 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.794791    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jts89"]
Jul 17 05:50:45.797270 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:50:45.797220    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ade4bd_5a9c_4a04_b311_ff623b15885d.slice/crio-ea508ba87cc8ed64f0afec2191004a4a1da0675fbdcff30ab3fe98ff9e4b922f WatchSource:0}: Error finding container ea508ba87cc8ed64f0afec2191004a4a1da0675fbdcff30ab3fe98ff9e4b922f: Status 404 returned error can't find the container with id ea508ba87cc8ed64f0afec2191004a4a1da0675fbdcff30ab3fe98ff9e4b922f
Jul 17 05:50:45.927249 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:45.927158    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jts89" event={"ID":"a1ade4bd-5a9c-4a04-b311-ff623b15885d","Type":"ContainerStarted","Data":"ea508ba87cc8ed64f0afec2191004a4a1da0675fbdcff30ab3fe98ff9e4b922f"}
Jul 17 05:50:46.075567 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:46.075537    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83a2d61b-6c06-4136-916c-01a82ff92723-tls-certs\") pod \"model-serving-api-54d89bd5fc-2tdtj\" (UID: \"83a2d61b-6c06-4136-916c-01a82ff92723\") " pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj"
Jul 17 05:50:46.078044 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:46.078016    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83a2d61b-6c06-4136-916c-01a82ff92723-tls-certs\") pod \"model-serving-api-54d89bd5fc-2tdtj\" (UID: \"83a2d61b-6c06-4136-916c-01a82ff92723\") " pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj"
Jul 17 05:50:46.246172 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:46.246091    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj"
Jul 17 05:50:46.369183 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:46.369115    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-2tdtj"]
Jul 17 05:50:46.371644 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:50:46.371616    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83a2d61b_6c06_4136_916c_01a82ff92723.slice/crio-641d520df7c1617c18336d281d79ee6350dc5fdf59373ca12acd3a3831177e0e WatchSource:0}: Error finding container 641d520df7c1617c18336d281d79ee6350dc5fdf59373ca12acd3a3831177e0e: Status 404 returned error can't find the container with id 641d520df7c1617c18336d281d79ee6350dc5fdf59373ca12acd3a3831177e0e
Jul 17 05:50:46.933776 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:46.933734    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj" event={"ID":"83a2d61b-6c06-4136-916c-01a82ff92723","Type":"ContainerStarted","Data":"641d520df7c1617c18336d281d79ee6350dc5fdf59373ca12acd3a3831177e0e"}
Jul 17 05:50:48.943816 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:48.943770    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj" event={"ID":"83a2d61b-6c06-4136-916c-01a82ff92723","Type":"ContainerStarted","Data":"134f468f516cdb49fdbfcc150c92ede2c630902e8dadb4f7305318e725355507"}
Jul 17 05:50:48.944240 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:48.943852    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj"
Jul 17 05:50:48.945345 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:48.945322    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jts89" event={"ID":"a1ade4bd-5a9c-4a04-b311-ff623b15885d","Type":"ContainerStarted","Data":"85c51c824c7838db95f5d3fb98dd447f3ba6cf32c95cf816a57cabc82ea07c17"}
Jul 17 05:50:48.945505 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:48.945488    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-jts89"
Jul 17 05:50:48.959397 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:48.959352    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj" podStartSLOduration=1.5610099499999999 podStartE2EDuration="3.959333735s" totalImagesPullingTime="2.398323785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:50:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:50:48.958181246 +0000 UTC m=+571.400119830" watchObservedRunningTime="2026-07-17 05:50:48.959333735 +0000 UTC m=+571.401272310"
Jul 17 05:50:48.975483 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:48.975427    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-jts89" podStartSLOduration=1.999992797 podStartE2EDuration="4.975414304s" totalImagesPullingTime="2.975421507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:50:48.974059296 +0000 UTC m=+571.415997870" watchObservedRunningTime="2026-07-17 05:50:48.975414304 +0000 UTC m=+571.417352877"
Jul 17 05:50:50.067215 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.067183    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"]
Jul 17 05:50:50.070397 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.070377    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"
Jul 17 05:50:50.073050 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.073025    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 17 05:50:50.073166 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.073122    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 17 05:50:50.073245 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.073229    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 05:50:50.074445 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.074422    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 05:50:50.074588 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.074568    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tnrsj\""
Jul 17 05:50:50.080208 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.080187    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"]
Jul 17 05:50:50.106267 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.106234    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tqb9\" (UniqueName: \"kubernetes.io/projected/c53de3f8-1f8f-4688-b4c2-5c21a48e41e9-kube-api-access-6tqb9\") pod \"kube-auth-proxy-79d55f4645-9qcsx\" (UID: \"c53de3f8-1f8f-4688-b4c2-5c21a48e41e9\") " pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"
Jul 17 05:50:50.106405 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.106343    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c53de3f8-1f8f-4688-b4c2-5c21a48e41e9-tmp\") pod \"kube-auth-proxy-79d55f4645-9qcsx\" (UID: \"c53de3f8-1f8f-4688-b4c2-5c21a48e41e9\") " pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"
Jul 17 05:50:50.106444 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.106411    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53de3f8-1f8f-4688-b4c2-5c21a48e41e9-tls-certs\") pod \"kube-auth-proxy-79d55f4645-9qcsx\" (UID: \"c53de3f8-1f8f-4688-b4c2-5c21a48e41e9\") " pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"
Jul 17 05:50:50.207552 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.207521    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c53de3f8-1f8f-4688-b4c2-5c21a48e41e9-tmp\") pod \"kube-auth-proxy-79d55f4645-9qcsx\" (UID: \"c53de3f8-1f8f-4688-b4c2-5c21a48e41e9\") " pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"
Jul 17 05:50:50.207736 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.207563    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53de3f8-1f8f-4688-b4c2-5c21a48e41e9-tls-certs\") pod \"kube-auth-proxy-79d55f4645-9qcsx\" (UID: \"c53de3f8-1f8f-4688-b4c2-5c21a48e41e9\") " pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"
Jul 17 05:50:50.207736 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.207616    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tqb9\" (UniqueName: \"kubernetes.io/projected/c53de3f8-1f8f-4688-b4c2-5c21a48e41e9-kube-api-access-6tqb9\") pod \"kube-auth-proxy-79d55f4645-9qcsx\" (UID: \"c53de3f8-1f8f-4688-b4c2-5c21a48e41e9\") " pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"
Jul 17 05:50:50.209841 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.209807    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c53de3f8-1f8f-4688-b4c2-5c21a48e41e9-tmp\") pod \"kube-auth-proxy-79d55f4645-9qcsx\" (UID: \"c53de3f8-1f8f-4688-b4c2-5c21a48e41e9\") " pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"
Jul 17 05:50:50.210054 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.210036    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53de3f8-1f8f-4688-b4c2-5c21a48e41e9-tls-certs\") pod \"kube-auth-proxy-79d55f4645-9qcsx\" (UID: \"c53de3f8-1f8f-4688-b4c2-5c21a48e41e9\") " pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"
Jul 17 05:50:50.216676 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.216650    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tqb9\" (UniqueName: \"kubernetes.io/projected/c53de3f8-1f8f-4688-b4c2-5c21a48e41e9-kube-api-access-6tqb9\") pod \"kube-auth-proxy-79d55f4645-9qcsx\" (UID: \"c53de3f8-1f8f-4688-b4c2-5c21a48e41e9\") " pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"
Jul 17 05:50:50.380020 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.379995    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"
Jul 17 05:50:50.509708 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.509682    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx"]
Jul 17 05:50:50.512031 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:50:50.512005    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53de3f8_1f8f_4688_b4c2_5c21a48e41e9.slice/crio-c37202725341f179aabb1f7650b8f0817d79ee9e0d3c2bd4eeffc13ba84ba0f6 WatchSource:0}: Error finding container c37202725341f179aabb1f7650b8f0817d79ee9e0d3c2bd4eeffc13ba84ba0f6: Status 404 returned error can't find the container with id c37202725341f179aabb1f7650b8f0817d79ee9e0d3c2bd4eeffc13ba84ba0f6
Jul 17 05:50:50.954623 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:50.954581    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx" event={"ID":"c53de3f8-1f8f-4688-b4c2-5c21a48e41e9","Type":"ContainerStarted","Data":"c37202725341f179aabb1f7650b8f0817d79ee9e0d3c2bd4eeffc13ba84ba0f6"}
Jul 17 05:50:51.079586 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.079551    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"]
Jul 17 05:50:51.084144 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.084114    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.087516 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.087491    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 17 05:50:51.087516 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.087507    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 05:50:51.087721 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.087509    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 17 05:50:51.087721 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.087510    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 17 05:50:51.087815 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.087780    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fcvqb\""
Jul 17 05:50:51.102240 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.102214    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"]
Jul 17 05:50:51.114075 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.114050    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a7e62eb-38ae-410e-a20b-1d84b484757a-local-certs\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.114183 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.114077    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.114183 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.114120    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9mck\" (UniqueName: \"kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-kube-api-access-z9mck\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.114262 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.114211    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.114262 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.114254    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.114327 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.114303    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-cacerts\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.114364 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.114352    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-token\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.214788 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.214706    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9mck\" (UniqueName: \"kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-kube-api-access-z9mck\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.214788 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.214757    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.214994 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.214820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.214994 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.214848    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-cacerts\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.214994 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.214882    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-token\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.214994 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.214968    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a7e62eb-38ae-410e-a20b-1d84b484757a-local-certs\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.214994 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.214993    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.215907 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.215883    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.217744 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.217718    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a7e62eb-38ae-410e-a20b-1d84b484757a-local-certs\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.217863 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.217820    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-cacerts\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.218011 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.217987    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.218153 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.218134    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.234852 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.234804    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9mck\" (UniqueName: \"kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-kube-api-access-z9mck\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.235502 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.235459    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-token\") pod \"istiod-openshift-gateway-7574459656-ldk8f\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.393790 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.393756    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:51.550562 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.550534    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"]
Jul 17 05:50:51.552813 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:50:51.552782    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a7e62eb_38ae_410e_a20b_1d84b484757a.slice/crio-54a708ad47d8671c5465c034b243b438810f10beca7683e538020ae7a7846b3f WatchSource:0}: Error finding container 54a708ad47d8671c5465c034b243b438810f10beca7683e538020ae7a7846b3f: Status 404 returned error can't find the container with id 54a708ad47d8671c5465c034b243b438810f10beca7683e538020ae7a7846b3f
Jul 17 05:50:51.961103 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:51.961046    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f" event={"ID":"8a7e62eb-38ae-410e-a20b-1d84b484757a","Type":"ContainerStarted","Data":"54a708ad47d8671c5465c034b243b438810f10beca7683e538020ae7a7846b3f"}
Jul 17 05:50:54.204596 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.204556    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 05:50:54.204906 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.204655    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 05:50:54.557740 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.557658    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"]
Jul 17 05:50:54.560441 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.560419    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"
Jul 17 05:50:54.563675 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.563656    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 17 05:50:54.563783 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.563675    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-82grc\""
Jul 17 05:50:54.588844 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.588821    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"]
Jul 17 05:50:54.590532 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.590513    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"]
Jul 17 05:50:54.593708 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.593693    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"
Jul 17 05:50:54.596240 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.596222    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 17 05:50:54.596341 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.596301    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-p5xsf\""
Jul 17 05:50:54.603618 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.603576    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"]
Jul 17 05:50:54.643392 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.643365    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzhb8\" (UniqueName: \"kubernetes.io/projected/4dd38d67-dd44-4f64-af29-e876a1c83eb1-kube-api-access-xzhb8\") pod \"kserve-controller-manager-d6f4b65b8-xzhjw\" (UID: \"4dd38d67-dd44-4f64-af29-e876a1c83eb1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"
Jul 17 05:50:54.643539 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.643399    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c74dbd36-746f-4a8f-bb08-6ab646a4836e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ztlql\" (UID: \"c74dbd36-746f-4a8f-bb08-6ab646a4836e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"
Jul 17 05:50:54.643539 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.643426    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll9jc\" (UniqueName: \"kubernetes.io/projected/c74dbd36-746f-4a8f-bb08-6ab646a4836e-kube-api-access-ll9jc\") pod \"llmisvc-controller-manager-777c6bd7d5-ztlql\" (UID: \"c74dbd36-746f-4a8f-bb08-6ab646a4836e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"
Jul 17 05:50:54.643642 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.643550    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4dd38d67-dd44-4f64-af29-e876a1c83eb1-cert\") pod \"kserve-controller-manager-d6f4b65b8-xzhjw\" (UID: \"4dd38d67-dd44-4f64-af29-e876a1c83eb1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"
Jul 17 05:50:54.744618 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.744595    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4dd38d67-dd44-4f64-af29-e876a1c83eb1-cert\") pod \"kserve-controller-manager-d6f4b65b8-xzhjw\" (UID: \"4dd38d67-dd44-4f64-af29-e876a1c83eb1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"
Jul 17 05:50:54.744701 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.744656    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzhb8\" (UniqueName: \"kubernetes.io/projected/4dd38d67-dd44-4f64-af29-e876a1c83eb1-kube-api-access-xzhb8\") pod \"kserve-controller-manager-d6f4b65b8-xzhjw\" (UID: \"4dd38d67-dd44-4f64-af29-e876a1c83eb1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"
Jul 17 05:50:54.744701 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.744686    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c74dbd36-746f-4a8f-bb08-6ab646a4836e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ztlql\" (UID: \"c74dbd36-746f-4a8f-bb08-6ab646a4836e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"
Jul 17 05:50:54.744796 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.744716    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll9jc\" (UniqueName: \"kubernetes.io/projected/c74dbd36-746f-4a8f-bb08-6ab646a4836e-kube-api-access-ll9jc\") pod \"llmisvc-controller-manager-777c6bd7d5-ztlql\" (UID: \"c74dbd36-746f-4a8f-bb08-6ab646a4836e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"
Jul 17 05:50:54.744796 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:50:54.744740    2558 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 17 05:50:54.744886 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:50:54.744802    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd38d67-dd44-4f64-af29-e876a1c83eb1-cert podName:4dd38d67-dd44-4f64-af29-e876a1c83eb1 nodeName:}" failed. No retries permitted until 2026-07-17 05:50:55.244783521 +0000 UTC m=+577.686722076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4dd38d67-dd44-4f64-af29-e876a1c83eb1-cert") pod "kserve-controller-manager-d6f4b65b8-xzhjw" (UID: "4dd38d67-dd44-4f64-af29-e876a1c83eb1") : secret "kserve-webhook-server-cert" not found
Jul 17 05:50:54.744886 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:50:54.744837    2558 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 17 05:50:54.744978 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:50:54.744896    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c74dbd36-746f-4a8f-bb08-6ab646a4836e-cert podName:c74dbd36-746f-4a8f-bb08-6ab646a4836e nodeName:}" failed. No retries permitted until 2026-07-17 05:50:55.244880566 +0000 UTC m=+577.686819120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c74dbd36-746f-4a8f-bb08-6ab646a4836e-cert") pod "llmisvc-controller-manager-777c6bd7d5-ztlql" (UID: "c74dbd36-746f-4a8f-bb08-6ab646a4836e") : secret "llmisvc-webhook-server-cert" not found
Jul 17 05:50:54.753711 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.753688    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzhb8\" (UniqueName: \"kubernetes.io/projected/4dd38d67-dd44-4f64-af29-e876a1c83eb1-kube-api-access-xzhb8\") pod \"kserve-controller-manager-d6f4b65b8-xzhjw\" (UID: \"4dd38d67-dd44-4f64-af29-e876a1c83eb1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"
Jul 17 05:50:54.753903 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.753884    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll9jc\" (UniqueName: \"kubernetes.io/projected/c74dbd36-746f-4a8f-bb08-6ab646a4836e-kube-api-access-ll9jc\") pod \"llmisvc-controller-manager-777c6bd7d5-ztlql\" (UID: \"c74dbd36-746f-4a8f-bb08-6ab646a4836e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"
Jul 17 05:50:54.977645 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.977612    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f" event={"ID":"8a7e62eb-38ae-410e-a20b-1d84b484757a","Type":"ContainerStarted","Data":"5cc03da7b2e41bba985f5bb8a1bdcf24f749681fbc330f3d12693189136a8b35"}
Jul 17 05:50:54.977832 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.977675    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:54.979103 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.979080    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx" event={"ID":"c53de3f8-1f8f-4688-b4c2-5c21a48e41e9","Type":"ContainerStarted","Data":"af6ad47903b7c0ca059ffd77ae3da9a67124c16528d82e410b7563dc6e7e7b01"}
Jul 17 05:50:54.997765 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:54.997721    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f" podStartSLOduration=1.3485797 podStartE2EDuration="3.997707254s" totalImagesPullingTime="2.649127554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:50:54.996364074 +0000 UTC m=+577.438302646" watchObservedRunningTime="2026-07-17 05:50:54.997707254 +0000 UTC m=+577.439645828"
Jul 17 05:50:55.024853 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.024797    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-79d55f4645-9qcsx" podStartSLOduration=1.3880384829999999 podStartE2EDuration="5.024778103s" totalImagesPullingTime="3.63673962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:50:55.016567677 +0000 UTC m=+577.458506251" watchObservedRunningTime="2026-07-17 05:50:55.024778103 +0000 UTC m=+577.466716678"
Jul 17 05:50:55.249958 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.249871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4dd38d67-dd44-4f64-af29-e876a1c83eb1-cert\") pod \"kserve-controller-manager-d6f4b65b8-xzhjw\" (UID: \"4dd38d67-dd44-4f64-af29-e876a1c83eb1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"
Jul 17 05:50:55.250342 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.249990    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c74dbd36-746f-4a8f-bb08-6ab646a4836e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ztlql\" (UID: \"c74dbd36-746f-4a8f-bb08-6ab646a4836e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"
Jul 17 05:50:55.252485 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.252439    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c74dbd36-746f-4a8f-bb08-6ab646a4836e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ztlql\" (UID: \"c74dbd36-746f-4a8f-bb08-6ab646a4836e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"
Jul 17 05:50:55.252608 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.252513    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4dd38d67-dd44-4f64-af29-e876a1c83eb1-cert\") pod \"kserve-controller-manager-d6f4b65b8-xzhjw\" (UID: \"4dd38d67-dd44-4f64-af29-e876a1c83eb1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"
Jul 17 05:50:55.470321 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.470274    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"
Jul 17 05:50:55.505150 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.505090    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"
Jul 17 05:50:55.610887 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.610784    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"]
Jul 17 05:50:55.613689 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:50:55.613657    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd38d67_dd44_4f64_af29_e876a1c83eb1.slice/crio-535773941b06118109d0ff663aded372f90e2647d3559e4683e753cb2273f589 WatchSource:0}: Error finding container 535773941b06118109d0ff663aded372f90e2647d3559e4683e753cb2273f589: Status 404 returned error can't find the container with id 535773941b06118109d0ff663aded372f90e2647d3559e4683e753cb2273f589
Jul 17 05:50:55.646789 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.646644    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"]
Jul 17 05:50:55.649142 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:50:55.649113    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc74dbd36_746f_4a8f_bb08_6ab646a4836e.slice/crio-7294a471abe51ec092ad9261765661531fcc403b45b72c67955d87f17d477995 WatchSource:0}: Error finding container 7294a471abe51ec092ad9261765661531fcc403b45b72c67955d87f17d477995: Status 404 returned error can't find the container with id 7294a471abe51ec092ad9261765661531fcc403b45b72c67955d87f17d477995
Jul 17 05:50:55.984161 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.984124    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw" event={"ID":"4dd38d67-dd44-4f64-af29-e876a1c83eb1","Type":"ContainerStarted","Data":"535773941b06118109d0ff663aded372f90e2647d3559e4683e753cb2273f589"}
Jul 17 05:50:55.985243 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.985217    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql" event={"ID":"c74dbd36-746f-4a8f-bb08-6ab646a4836e","Type":"ContainerStarted","Data":"7294a471abe51ec092ad9261765661531fcc403b45b72c67955d87f17d477995"}
Jul 17 05:50:55.986851 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.986828    2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-ldk8f container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 05:50:55.986935 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:55.986874    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f" podUID="8a7e62eb-38ae-410e-a20b-1d84b484757a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:50:56.138096 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.138063    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"]
Jul 17 05:50:56.143091 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.143066    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.146270 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.146250    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qj77n\""
Jul 17 05:50:56.154247 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.154228    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"]
Jul 17 05:50:56.158447 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.158427    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.158551 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.158494    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.158551 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.158533    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.158662 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.158559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.158662 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.158607    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.158761 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.158657    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4mh9\" (UniqueName: \"kubernetes.io/projected/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-kube-api-access-k4mh9\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.158761 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.158692    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.158761 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.158738    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.158890 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.158763    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.259763 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.259693    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.259763 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.259730    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.259763 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.259747    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.260239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.259775    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.260239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.259932    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.260239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.259998    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.260239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.260045    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.260239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.260069    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.260239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.260107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4mh9\" (UniqueName: \"kubernetes.io/projected/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-kube-api-access-k4mh9\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.260239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.260112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.260239 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.260207    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.260588 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.260287    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.260643 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.260585    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.260694 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.260646    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.262666 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.262642    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.262808 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.262779    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.296432 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.296396    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.296815 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.296798    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4mh9\" (UniqueName: \"kubernetes.io/projected/a21ac819-78e6-4b7e-abbd-616e6bf6fb37-kube-api-access-k4mh9\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42j8sn\" (UID: \"a21ac819-78e6-4b7e-abbd-616e6bf6fb37\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.455370 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.455341    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:50:56.631092 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.631066    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"]
Jul 17 05:50:56.636978 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:50:56.636941    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda21ac819_78e6_4b7e_abbd_616e6bf6fb37.slice/crio-4e74de874f21c841660d3327f529894a69212f60256e0dd44bab75907cd8b2df WatchSource:0}: Error finding container 4e74de874f21c841660d3327f529894a69212f60256e0dd44bab75907cd8b2df: Status 404 returned error can't find the container with id 4e74de874f21c841660d3327f529894a69212f60256e0dd44bab75907cd8b2df
Jul 17 05:50:56.991301 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:56.991270    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn" event={"ID":"a21ac819-78e6-4b7e-abbd-616e6bf6fb37","Type":"ContainerStarted","Data":"4e74de874f21c841660d3327f529894a69212f60256e0dd44bab75907cd8b2df"}
Jul 17 05:50:57.998093 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:57.997747    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"]
Jul 17 05:50:57.998093 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:57.997947    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f" podUID="8a7e62eb-38ae-410e-a20b-1d84b484757a" containerName="discovery" containerID="cri-o://5cc03da7b2e41bba985f5bb8a1bdcf24f749681fbc330f3d12693189136a8b35" gracePeriod=30
Jul 17 05:50:57.998928 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:57.998754    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:59.006557 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.006520    2558 generic.go:356] "Generic (PLEG): container finished" podID="8a7e62eb-38ae-410e-a20b-1d84b484757a" containerID="5cc03da7b2e41bba985f5bb8a1bdcf24f749681fbc330f3d12693189136a8b35" exitCode=0
Jul 17 05:50:59.006981 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.006610    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f" event={"ID":"8a7e62eb-38ae-410e-a20b-1d84b484757a","Type":"ContainerDied","Data":"5cc03da7b2e41bba985f5bb8a1bdcf24f749681fbc330f3d12693189136a8b35"}
Jul 17 05:50:59.243316 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.243294    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:50:59.284269 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.284181    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a7e62eb-38ae-410e-a20b-1d84b484757a-local-certs\") pod \"8a7e62eb-38ae-410e-a20b-1d84b484757a\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") "
Jul 17 05:50:59.284269 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.284233    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9mck\" (UniqueName: \"kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-kube-api-access-z9mck\") pod \"8a7e62eb-38ae-410e-a20b-1d84b484757a\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") "
Jul 17 05:50:59.284269 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.284260    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-ca-configmap\") pod \"8a7e62eb-38ae-410e-a20b-1d84b484757a\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") "
Jul 17 05:50:59.284610 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.284322    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-dns-cert\") pod \"8a7e62eb-38ae-410e-a20b-1d84b484757a\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") "
Jul 17 05:50:59.284610 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.284376    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-token\") pod \"8a7e62eb-38ae-410e-a20b-1d84b484757a\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") "
Jul 17 05:50:59.284610 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.284432    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-kubeconfig\") pod \"8a7e62eb-38ae-410e-a20b-1d84b484757a\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") "
Jul 17 05:50:59.284610 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.284463    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-cacerts\") pod \"8a7e62eb-38ae-410e-a20b-1d84b484757a\" (UID: \"8a7e62eb-38ae-410e-a20b-1d84b484757a\") "
Jul 17 05:50:59.285411 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.285380    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "8a7e62eb-38ae-410e-a20b-1d84b484757a" (UID: "8a7e62eb-38ae-410e-a20b-1d84b484757a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:50:59.287536 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.287499    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7e62eb-38ae-410e-a20b-1d84b484757a-local-certs" (OuterVolumeSpecName: "local-certs") pod "8a7e62eb-38ae-410e-a20b-1d84b484757a" (UID: "8a7e62eb-38ae-410e-a20b-1d84b484757a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:50:59.287735 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.287716    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-kube-api-access-z9mck" (OuterVolumeSpecName: "kube-api-access-z9mck") pod "8a7e62eb-38ae-410e-a20b-1d84b484757a" (UID: "8a7e62eb-38ae-410e-a20b-1d84b484757a"). InnerVolumeSpecName "kube-api-access-z9mck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:50:59.287735 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.287718    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-cacerts" (OuterVolumeSpecName: "cacerts") pod "8a7e62eb-38ae-410e-a20b-1d84b484757a" (UID: "8a7e62eb-38ae-410e-a20b-1d84b484757a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:50:59.287964 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.287943    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-token" (OuterVolumeSpecName: "istio-token") pod "8a7e62eb-38ae-410e-a20b-1d84b484757a" (UID: "8a7e62eb-38ae-410e-a20b-1d84b484757a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:50:59.288104 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.288081    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "8a7e62eb-38ae-410e-a20b-1d84b484757a" (UID: "8a7e62eb-38ae-410e-a20b-1d84b484757a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:50:59.288300 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.288279    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "8a7e62eb-38ae-410e-a20b-1d84b484757a" (UID: "8a7e62eb-38ae-410e-a20b-1d84b484757a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:50:59.385803 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.385771    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:50:59.385803 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.385796    2558 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-kubeconfig\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:50:59.385803 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.385806    2558 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-cacerts\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:50:59.386073 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.385828    2558 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a7e62eb-38ae-410e-a20b-1d84b484757a-local-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:50:59.386073 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.385839    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9mck\" (UniqueName: \"kubernetes.io/projected/8a7e62eb-38ae-410e-a20b-1d84b484757a-kube-api-access-z9mck\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:50:59.386073 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.385853    2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-ca-configmap\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:50:59.386073 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.385862    2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8a7e62eb-38ae-410e-a20b-1d84b484757a-istio-csr-dns-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:50:59.809378 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.809349    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 05:50:59.809445 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.809432    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 05:50:59.809504 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.809464    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 05:50:59.952041 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.952006    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-jts89"
Jul 17 05:50:59.955692 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:50:59.955673    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-2tdtj"
Jul 17 05:51:00.014116 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.013317    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql" event={"ID":"c74dbd36-746f-4a8f-bb08-6ab646a4836e","Type":"ContainerStarted","Data":"367425e944cc1533d67f2da59f533091ce9367bde9a762c648ab2b4efdd87cce"}
Jul 17 05:51:00.014116 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.013766    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"
Jul 17 05:51:00.015305 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.015271    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f" event={"ID":"8a7e62eb-38ae-410e-a20b-1d84b484757a","Type":"ContainerDied","Data":"54a708ad47d8671c5465c034b243b438810f10beca7683e538020ae7a7846b3f"}
Jul 17 05:51:00.015429 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.015278    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"
Jul 17 05:51:00.015429 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.015319    2558 scope.go:117] "RemoveContainer" containerID="5cc03da7b2e41bba985f5bb8a1bdcf24f749681fbc330f3d12693189136a8b35"
Jul 17 05:51:00.017906 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.017854    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn" event={"ID":"a21ac819-78e6-4b7e-abbd-616e6bf6fb37","Type":"ContainerStarted","Data":"1e849a35fda8c41b0fccf26ef363b644928ae09791c74303a1d6dfe2e1c4442d"}
Jul 17 05:51:00.019670 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.019629    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw" event={"ID":"4dd38d67-dd44-4f64-af29-e876a1c83eb1","Type":"ContainerStarted","Data":"3e82f3d577c03602c306126f88bc70269d6a7ad0877b91084b428cf8810bdaa2"}
Jul 17 05:51:00.019873 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.019850    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"
Jul 17 05:51:00.033271 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.033157    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql" podStartSLOduration=1.8356824170000001 podStartE2EDuration="6.033140572s" totalImagesPullingTime="4.197458155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:50:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:51:00.032579192 +0000 UTC m=+582.474517787" watchObservedRunningTime="2026-07-17 05:51:00.033140572 +0000 UTC m=+582.475079148"
Jul 17 05:51:00.049964 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.049921    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw" podStartSLOduration=1.860276604 podStartE2EDuration="6.049908503s" totalImagesPullingTime="4.189631899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:50:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:51:00.048825275 +0000 UTC m=+582.490763871" watchObservedRunningTime="2026-07-17 05:51:00.049908503 +0000 UTC m=+582.491847103"
Jul 17 05:51:00.067753 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.067676    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn" podStartSLOduration=0.898077995 podStartE2EDuration="4.067662045s" totalImagesPullingTime="3.16958405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:50:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:51:00.065939079 +0000 UTC m=+582.507877671" watchObservedRunningTime="2026-07-17 05:51:00.067662045 +0000 UTC m=+582.509600617"
Jul 17 05:51:00.084601 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.084578    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"]
Jul 17 05:51:00.086718 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.086698    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-ldk8f"]
Jul 17 05:51:00.456330 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.456294    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:51:00.457793 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.457764    2558 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6576859cf42j8sn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Jul 17 05:51:00.457917 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:00.457829    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn" podUID="a21ac819-78e6-4b7e-abbd-616e6bf6fb37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Jul 17 05:51:01.459065 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:01.459038    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:51:02.031211 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:02.031180    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:51:02.032133 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:02.032110    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42j8sn"
Jul 17 05:51:02.084143 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:02.080965    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a7e62eb-38ae-410e-a20b-1d84b484757a" path="/var/lib/kubelet/pods/8a7e62eb-38ae-410e-a20b-1d84b484757a/volumes"
Jul 17 05:51:17.949816 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:17.949794    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 05:51:17.950092 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:17.949899    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 05:51:17.954379 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:17.954359    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 05:51:17.954511 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:17.954496    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 05:51:31.028739 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:31.028710    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ztlql"
Jul 17 05:51:31.056511 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:31.031914    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xzhjw"
Jul 17 05:51:45.597308 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.597211    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bmvpr"]
Jul 17 05:51:45.598264 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.597885    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7e62eb-38ae-410e-a20b-1d84b484757a" containerName="discovery"
Jul 17 05:51:45.598264 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.597904    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7e62eb-38ae-410e-a20b-1d84b484757a" containerName="discovery"
Jul 17 05:51:45.598264 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.598005    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a7e62eb-38ae-410e-a20b-1d84b484757a" containerName="discovery"
Jul 17 05:51:45.600281 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.600262    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bmvpr"
Jul 17 05:51:45.603185 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.603153    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 05:51:45.604331 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.604314    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 05:51:45.604429 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.604355    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-vw86h\""
Jul 17 05:51:45.608334 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.608249    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bmvpr"]
Jul 17 05:51:45.739553 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.739516    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fldr\" (UniqueName: \"kubernetes.io/projected/36aa724c-7881-4c61-9eda-a94a558aab3d-kube-api-access-9fldr\") pod \"kuadrant-operator-catalog-bmvpr\" (UID: \"36aa724c-7881-4c61-9eda-a94a558aab3d\") " pod="kuadrant-system/kuadrant-operator-catalog-bmvpr"
Jul 17 05:51:45.840156 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.840126    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fldr\" (UniqueName: \"kubernetes.io/projected/36aa724c-7881-4c61-9eda-a94a558aab3d-kube-api-access-9fldr\") pod \"kuadrant-operator-catalog-bmvpr\" (UID: \"36aa724c-7881-4c61-9eda-a94a558aab3d\") " pod="kuadrant-system/kuadrant-operator-catalog-bmvpr"
Jul 17 05:51:45.857761 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.857701    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fldr\" (UniqueName: \"kubernetes.io/projected/36aa724c-7881-4c61-9eda-a94a558aab3d-kube-api-access-9fldr\") pod \"kuadrant-operator-catalog-bmvpr\" (UID: \"36aa724c-7881-4c61-9eda-a94a558aab3d\") " pod="kuadrant-system/kuadrant-operator-catalog-bmvpr"
Jul 17 05:51:45.911130 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.911104    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bmvpr"
Jul 17 05:51:45.963136 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:45.963106    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bmvpr"]
Jul 17 05:51:46.035491 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:46.035452    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bmvpr"]
Jul 17 05:51:46.038814 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:51:46.038788    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36aa724c_7881_4c61_9eda_a94a558aab3d.slice/crio-0aea04921131bff025a1f6997e8fc1c467ea706dbed3230d3ff46769812cb369 WatchSource:0}: Error finding container 0aea04921131bff025a1f6997e8fc1c467ea706dbed3230d3ff46769812cb369: Status 404 returned error can't find the container with id 0aea04921131bff025a1f6997e8fc1c467ea706dbed3230d3ff46769812cb369
Jul 17 05:51:46.171717 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:46.171643    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rlnmc"]
Jul 17 05:51:46.174597 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:46.174582    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rlnmc"
Jul 17 05:51:46.186334 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:46.186312    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rlnmc"]
Jul 17 05:51:46.209681 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:46.209652    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bmvpr" event={"ID":"36aa724c-7881-4c61-9eda-a94a558aab3d","Type":"ContainerStarted","Data":"0aea04921131bff025a1f6997e8fc1c467ea706dbed3230d3ff46769812cb369"}
Jul 17 05:51:46.343944 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:46.343912    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6cgj\" (UniqueName: \"kubernetes.io/projected/9196e4da-33d8-4e46-acb8-411d0e81cd8c-kube-api-access-g6cgj\") pod \"kuadrant-operator-catalog-rlnmc\" (UID: \"9196e4da-33d8-4e46-acb8-411d0e81cd8c\") " pod="kuadrant-system/kuadrant-operator-catalog-rlnmc"
Jul 17 05:51:46.444801 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:46.444716    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6cgj\" (UniqueName: \"kubernetes.io/projected/9196e4da-33d8-4e46-acb8-411d0e81cd8c-kube-api-access-g6cgj\") pod \"kuadrant-operator-catalog-rlnmc\" (UID: \"9196e4da-33d8-4e46-acb8-411d0e81cd8c\") " pod="kuadrant-system/kuadrant-operator-catalog-rlnmc"
Jul 17 05:51:46.452944 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:46.452916    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6cgj\" (UniqueName: \"kubernetes.io/projected/9196e4da-33d8-4e46-acb8-411d0e81cd8c-kube-api-access-g6cgj\") pod \"kuadrant-operator-catalog-rlnmc\" (UID: \"9196e4da-33d8-4e46-acb8-411d0e81cd8c\") " pod="kuadrant-system/kuadrant-operator-catalog-rlnmc"
Jul 17 05:51:46.484094 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:46.484072    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rlnmc"
Jul 17 05:51:46.611034 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:46.611013    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rlnmc"]
Jul 17 05:51:46.612706 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:51:46.612676    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9196e4da_33d8_4e46_acb8_411d0e81cd8c.slice/crio-a4dffaac33ba3b13a902f0fa60f9e9d4cd57efbb662e5da070cc8dce14cfe038 WatchSource:0}: Error finding container a4dffaac33ba3b13a902f0fa60f9e9d4cd57efbb662e5da070cc8dce14cfe038: Status 404 returned error can't find the container with id a4dffaac33ba3b13a902f0fa60f9e9d4cd57efbb662e5da070cc8dce14cfe038
Jul 17 05:51:47.218658 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:47.218604    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rlnmc" event={"ID":"9196e4da-33d8-4e46-acb8-411d0e81cd8c","Type":"ContainerStarted","Data":"a4dffaac33ba3b13a902f0fa60f9e9d4cd57efbb662e5da070cc8dce14cfe038"}
Jul 17 05:51:48.225649 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:48.225603    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rlnmc" event={"ID":"9196e4da-33d8-4e46-acb8-411d0e81cd8c","Type":"ContainerStarted","Data":"b55d1066b8469f2020bd9dc641683850f1cfcd95504ce53005b1fd1c316996cc"}
Jul 17 05:51:48.227294 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:48.227272    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bmvpr" event={"ID":"36aa724c-7881-4c61-9eda-a94a558aab3d","Type":"ContainerStarted","Data":"5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af"}
Jul 17 05:51:48.227404 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:48.227354    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-bmvpr" podUID="36aa724c-7881-4c61-9eda-a94a558aab3d" containerName="registry-server" containerID="cri-o://5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af" gracePeriod=2
Jul 17 05:51:48.241868 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:48.241820    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rlnmc" podStartSLOduration=0.770850713 podStartE2EDuration="2.241806479s" totalImagesPullingTime="1.470955766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:51:48.240185733 +0000 UTC m=+630.682124307" watchObservedRunningTime="2026-07-17 05:51:48.241806479 +0000 UTC m=+630.683745053"
Jul 17 05:51:48.255942 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:48.255903    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-bmvpr" podStartSLOduration=1.236866392 podStartE2EDuration="3.255888997s" totalImagesPullingTime="2.019022605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:51:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:51:48.254670161 +0000 UTC m=+630.696608736" watchObservedRunningTime="2026-07-17 05:51:48.255888997 +0000 UTC m=+630.697827570"
Jul 17 05:51:48.468355 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:48.468328    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bmvpr"
Jul 17 05:51:48.563957 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:48.563922    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fldr\" (UniqueName: \"kubernetes.io/projected/36aa724c-7881-4c61-9eda-a94a558aab3d-kube-api-access-9fldr\") pod \"36aa724c-7881-4c61-9eda-a94a558aab3d\" (UID: \"36aa724c-7881-4c61-9eda-a94a558aab3d\") "
Jul 17 05:51:48.566086 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:48.566064    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36aa724c-7881-4c61-9eda-a94a558aab3d-kube-api-access-9fldr" (OuterVolumeSpecName: "kube-api-access-9fldr") pod "36aa724c-7881-4c61-9eda-a94a558aab3d" (UID: "36aa724c-7881-4c61-9eda-a94a558aab3d"). InnerVolumeSpecName "kube-api-access-9fldr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:51:48.664734 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:48.664661    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fldr\" (UniqueName: \"kubernetes.io/projected/36aa724c-7881-4c61-9eda-a94a558aab3d-kube-api-access-9fldr\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:51:49.233173 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:49.233138    2558 generic.go:356] "Generic (PLEG): container finished" podID="36aa724c-7881-4c61-9eda-a94a558aab3d" containerID="5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af" exitCode=0
Jul 17 05:51:49.233634 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:49.233198    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bmvpr"
Jul 17 05:51:49.233634 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:49.233219    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bmvpr" event={"ID":"36aa724c-7881-4c61-9eda-a94a558aab3d","Type":"ContainerDied","Data":"5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af"}
Jul 17 05:51:49.233634 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:49.233265    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bmvpr" event={"ID":"36aa724c-7881-4c61-9eda-a94a558aab3d","Type":"ContainerDied","Data":"0aea04921131bff025a1f6997e8fc1c467ea706dbed3230d3ff46769812cb369"}
Jul 17 05:51:49.233634 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:49.233286    2558 scope.go:117] "RemoveContainer" containerID="5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af"
Jul 17 05:51:49.242613 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:49.242595    2558 scope.go:117] "RemoveContainer" containerID="5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af"
Jul 17 05:51:49.242849 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:51:49.242833    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af\": container with ID starting with 5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af not found: ID does not exist" containerID="5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af"
Jul 17 05:51:49.242912 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:49.242858    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af"} err="failed to get container status \"5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af\": rpc error: code = NotFound desc = could not find container \"5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af\": container with ID starting with 5e6f66b4b81dc2c96498e2e7e32d79ae9b88a0f87e2f3c9f35a13cf4b25fb7af not found: ID does not exist"
Jul 17 05:51:49.253029 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:49.252996    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bmvpr"]
Jul 17 05:51:49.256964 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:49.256944    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bmvpr"]
Jul 17 05:51:50.077562 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:50.077531    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36aa724c-7881-4c61-9eda-a94a558aab3d" path="/var/lib/kubelet/pods/36aa724c-7881-4c61-9eda-a94a558aab3d/volumes"
Jul 17 05:51:56.484782 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:56.484749    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-rlnmc"
Jul 17 05:51:56.485235 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:56.484811    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-rlnmc"
Jul 17 05:51:56.506605 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:56.506583    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-rlnmc"
Jul 17 05:51:57.290262 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:51:57.290237    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-rlnmc"
Jul 17 05:52:17.025097 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.025065    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"]
Jul 17 05:52:17.025557 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.025386    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36aa724c-7881-4c61-9eda-a94a558aab3d" containerName="registry-server"
Jul 17 05:52:17.025557 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.025396    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="36aa724c-7881-4c61-9eda-a94a558aab3d" containerName="registry-server"
Jul 17 05:52:17.025557 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.025455    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="36aa724c-7881-4c61-9eda-a94a558aab3d" containerName="registry-server"
Jul 17 05:52:17.027453 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.027438    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"
Jul 17 05:52:17.030594 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.030572    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-sb62l\""
Jul 17 05:52:17.038770 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.038750    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"]
Jul 17 05:52:17.086938 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.086913    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm2ck\" (UniqueName: \"kubernetes.io/projected/48c22755-698f-4474-84af-39fed41e1b32-kube-api-access-vm2ck\") pod \"limitador-operator-controller-manager-859d7b78bd-4qglp\" (UID: \"48c22755-698f-4474-84af-39fed41e1b32\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"
Jul 17 05:52:17.188224 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.188203    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm2ck\" (UniqueName: \"kubernetes.io/projected/48c22755-698f-4474-84af-39fed41e1b32-kube-api-access-vm2ck\") pod \"limitador-operator-controller-manager-859d7b78bd-4qglp\" (UID: \"48c22755-698f-4474-84af-39fed41e1b32\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"
Jul 17 05:52:17.196688 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.196667    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm2ck\" (UniqueName: \"kubernetes.io/projected/48c22755-698f-4474-84af-39fed41e1b32-kube-api-access-vm2ck\") pod \"limitador-operator-controller-manager-859d7b78bd-4qglp\" (UID: \"48c22755-698f-4474-84af-39fed41e1b32\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"
Jul 17 05:52:17.337183 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.337152    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"
Jul 17 05:52:17.479283 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:17.479260    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"]
Jul 17 05:52:17.481920 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:52:17.481892    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c22755_698f_4474_84af_39fed41e1b32.slice/crio-e367854fafcbe74fed6d8f62dac8550f1d6e4531ef2fe7664e0a1e6e2025234d WatchSource:0}: Error finding container e367854fafcbe74fed6d8f62dac8550f1d6e4531ef2fe7664e0a1e6e2025234d: Status 404 returned error can't find the container with id e367854fafcbe74fed6d8f62dac8550f1d6e4531ef2fe7664e0a1e6e2025234d
Jul 17 05:52:18.364557 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:18.364514    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp" event={"ID":"48c22755-698f-4474-84af-39fed41e1b32","Type":"ContainerStarted","Data":"e367854fafcbe74fed6d8f62dac8550f1d6e4531ef2fe7664e0a1e6e2025234d"}
Jul 17 05:52:19.370111 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:19.370030    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp" event={"ID":"48c22755-698f-4474-84af-39fed41e1b32","Type":"ContainerStarted","Data":"55a385ba47c5b161ba4ae9d77d3212c59e75e84082950e96f7a59d11d2db7b06"}
Jul 17 05:52:19.370438 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:19.370117    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"
Jul 17 05:52:19.390925 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:19.390878    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp" podStartSLOduration=1.790258691 podStartE2EDuration="3.390863484s" totalImagesPullingTime="1.600604793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:52:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:52:19.390321531 +0000 UTC m=+661.832260137" watchObservedRunningTime="2026-07-17 05:52:19.390863484 +0000 UTC m=+661.832802057"
Jul 17 05:52:21.098281 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:21.098253    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wpslm"]
Jul 17 05:52:21.100587 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:21.100569    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wpslm"
Jul 17 05:52:21.103344 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:21.103324    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-fn5tm\""
Jul 17 05:52:21.113416 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:21.113389    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wpslm"]
Jul 17 05:52:21.221332 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:21.221308    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wbq8\" (UniqueName: \"kubernetes.io/projected/4a3fadb5-1911-42c0-b52e-c3834d59cd41-kube-api-access-6wbq8\") pod \"authorino-operator-75cc8d5cb7-wpslm\" (UID: \"4a3fadb5-1911-42c0-b52e-c3834d59cd41\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wpslm"
Jul 17 05:52:21.321840 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:21.321811    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wbq8\" (UniqueName: \"kubernetes.io/projected/4a3fadb5-1911-42c0-b52e-c3834d59cd41-kube-api-access-6wbq8\") pod \"authorino-operator-75cc8d5cb7-wpslm\" (UID: \"4a3fadb5-1911-42c0-b52e-c3834d59cd41\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wpslm"
Jul 17 05:52:21.337064 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:21.337036    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wbq8\" (UniqueName: \"kubernetes.io/projected/4a3fadb5-1911-42c0-b52e-c3834d59cd41-kube-api-access-6wbq8\") pod \"authorino-operator-75cc8d5cb7-wpslm\" (UID: \"4a3fadb5-1911-42c0-b52e-c3834d59cd41\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wpslm"
Jul 17 05:52:21.410769 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:21.410714    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wpslm"
Jul 17 05:52:21.545781 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:21.545759    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wpslm"]
Jul 17 05:52:21.548372 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:52:21.548337    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3fadb5_1911_42c0_b52e_c3834d59cd41.slice/crio-a85b9a60ce143de8562786b2fd6802c1946c3220474a3735cd259a98548d08ec WatchSource:0}: Error finding container a85b9a60ce143de8562786b2fd6802c1946c3220474a3735cd259a98548d08ec: Status 404 returned error can't find the container with id a85b9a60ce143de8562786b2fd6802c1946c3220474a3735cd259a98548d08ec
Jul 17 05:52:22.386357 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:22.386322    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wpslm" event={"ID":"4a3fadb5-1911-42c0-b52e-c3834d59cd41","Type":"ContainerStarted","Data":"a85b9a60ce143de8562786b2fd6802c1946c3220474a3735cd259a98548d08ec"}
Jul 17 05:52:24.397563 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:24.397527    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wpslm" event={"ID":"4a3fadb5-1911-42c0-b52e-c3834d59cd41","Type":"ContainerStarted","Data":"bb68fcb360e8f89321010692ee397faf5a55db2012c40be27caf651bceb75a9e"}
Jul 17 05:52:24.397965 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:24.397630    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wpslm"
Jul 17 05:52:24.413074 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:24.413030    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wpslm" podStartSLOduration=1.261045393 podStartE2EDuration="3.41301686s" totalImagesPullingTime="2.151971467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:52:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:52:24.4122993 +0000 UTC m=+666.854237873" watchObservedRunningTime="2026-07-17 05:52:24.41301686 +0000 UTC m=+666.854955433"
Jul 17 05:52:26.078579 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.078554    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"]
Jul 17 05:52:26.080849 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.080834    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:26.083633 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.083613    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vgf6m\""
Jul 17 05:52:26.090516 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.090496    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"]
Jul 17 05:52:26.159462 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.159432    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5bnl\" (UniqueName: \"kubernetes.io/projected/babed64b-c70c-44fe-96c5-f5aedfa52efc-kube-api-access-h5bnl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h27xf\" (UID: \"babed64b-c70c-44fe-96c5-f5aedfa52efc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:26.159607 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.159531    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/babed64b-c70c-44fe-96c5-f5aedfa52efc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h27xf\" (UID: \"babed64b-c70c-44fe-96c5-f5aedfa52efc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:26.260180 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.260153    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5bnl\" (UniqueName: \"kubernetes.io/projected/babed64b-c70c-44fe-96c5-f5aedfa52efc-kube-api-access-h5bnl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h27xf\" (UID: \"babed64b-c70c-44fe-96c5-f5aedfa52efc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:26.260279 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.260196    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/babed64b-c70c-44fe-96c5-f5aedfa52efc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h27xf\" (UID: \"babed64b-c70c-44fe-96c5-f5aedfa52efc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:26.260489 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.260453    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/babed64b-c70c-44fe-96c5-f5aedfa52efc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h27xf\" (UID: \"babed64b-c70c-44fe-96c5-f5aedfa52efc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:26.270973 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.270947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5bnl\" (UniqueName: \"kubernetes.io/projected/babed64b-c70c-44fe-96c5-f5aedfa52efc-kube-api-access-h5bnl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h27xf\" (UID: \"babed64b-c70c-44fe-96c5-f5aedfa52efc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:26.390978 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.390950    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:26.524113 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:26.524089    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"]
Jul 17 05:52:26.526367 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:52:26.526335    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbabed64b_c70c_44fe_96c5_f5aedfa52efc.slice/crio-c96559a5c428141f20d3b444ce913622c1e6868f1139cd321c5a4fd2ad02498a WatchSource:0}: Error finding container c96559a5c428141f20d3b444ce913622c1e6868f1139cd321c5a4fd2ad02498a: Status 404 returned error can't find the container with id c96559a5c428141f20d3b444ce913622c1e6868f1139cd321c5a4fd2ad02498a
Jul 17 05:52:27.415197 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:27.415154    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf" event={"ID":"babed64b-c70c-44fe-96c5-f5aedfa52efc","Type":"ContainerStarted","Data":"c96559a5c428141f20d3b444ce913622c1e6868f1139cd321c5a4fd2ad02498a"}
Jul 17 05:52:29.214256 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:29.214220    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w"]
Jul 17 05:52:29.216654 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:29.216639    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w"
Jul 17 05:52:29.219522 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:29.219497    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-zxvzk\""
Jul 17 05:52:29.219635 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:29.219554    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 17 05:52:29.229307 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:29.229282    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w"]
Jul 17 05:52:29.291900 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:29.291872    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djbcn\" (UniqueName: \"kubernetes.io/projected/35abfa32-e9ef-4329-8e20-65b88a06c027-kube-api-access-djbcn\") pod \"dns-operator-controller-manager-864dd78c4c-47p8w\" (UID: \"35abfa32-e9ef-4329-8e20-65b88a06c027\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w"
Jul 17 05:52:29.392939 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:29.392913    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djbcn\" (UniqueName: \"kubernetes.io/projected/35abfa32-e9ef-4329-8e20-65b88a06c027-kube-api-access-djbcn\") pod \"dns-operator-controller-manager-864dd78c4c-47p8w\" (UID: \"35abfa32-e9ef-4329-8e20-65b88a06c027\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w"
Jul 17 05:52:29.407217 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:29.407184    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djbcn\" (UniqueName: \"kubernetes.io/projected/35abfa32-e9ef-4329-8e20-65b88a06c027-kube-api-access-djbcn\") pod \"dns-operator-controller-manager-864dd78c4c-47p8w\" (UID: \"35abfa32-e9ef-4329-8e20-65b88a06c027\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w"
Jul 17 05:52:29.526798 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:29.526742    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w"
Jul 17 05:52:29.665718 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:29.665687    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w"]
Jul 17 05:52:29.669042 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:52:29.669016    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35abfa32_e9ef_4329_8e20_65b88a06c027.slice/crio-a0ae10c5abb58443f0fd957917c3e5e4079052f744f2eeff059cf19b4d2a0415 WatchSource:0}: Error finding container a0ae10c5abb58443f0fd957917c3e5e4079052f744f2eeff059cf19b4d2a0415: Status 404 returned error can't find the container with id a0ae10c5abb58443f0fd957917c3e5e4079052f744f2eeff059cf19b4d2a0415
Jul 17 05:52:30.377398 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:30.377368    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"
Jul 17 05:52:30.430159 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:30.430133    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w" event={"ID":"35abfa32-e9ef-4329-8e20-65b88a06c027","Type":"ContainerStarted","Data":"a0ae10c5abb58443f0fd957917c3e5e4079052f744f2eeff059cf19b4d2a0415"}
Jul 17 05:52:33.448048 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:33.448010    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf" event={"ID":"babed64b-c70c-44fe-96c5-f5aedfa52efc","Type":"ContainerStarted","Data":"c7675f9d37eb5676cfbc93fb863287c8956eb23bc2b4ccbeb10902e0a5856c4e"}
Jul 17 05:52:33.448598 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:33.448081    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:33.449503 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:33.449457    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w" event={"ID":"35abfa32-e9ef-4329-8e20-65b88a06c027","Type":"ContainerStarted","Data":"ce7454a3db71ada094b60f6d9e0b52aa9829afcfaa28f0ae02c9beb86cc659d0"}
Jul 17 05:52:33.449591 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:33.449527    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w"
Jul 17 05:52:33.469659 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:33.469620    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf" podStartSLOduration=1.4450623949999999 podStartE2EDuration="7.469606833s" totalImagesPullingTime="6.024544438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:52:33.468328022 +0000 UTC m=+675.910266618" watchObservedRunningTime="2026-07-17 05:52:33.469606833 +0000 UTC m=+675.911545406"
Jul 17 05:52:33.485417 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:33.485376    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w" podStartSLOduration=1.5490851220000001 podStartE2EDuration="4.48536128s" totalImagesPullingTime="2.936276158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:52:33.483841332 +0000 UTC m=+675.925779929" watchObservedRunningTime="2026-07-17 05:52:33.48536128 +0000 UTC m=+675.927299853"
Jul 17 05:52:35.156646 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.156612    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"]
Jul 17 05:52:35.160155 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.160137    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:35.162680 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.162654    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 17 05:52:35.162787 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.162690    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cksw5\""
Jul 17 05:52:35.162787 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.162699    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 17 05:52:35.170894 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.170875    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"]
Jul 17 05:52:35.248341 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.248315    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bde81c2f-1b0b-4adb-b100-abad3ec29772-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pxsl9\" (UID: \"bde81c2f-1b0b-4adb-b100-abad3ec29772\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:35.248432 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.248358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde81c2f-1b0b-4adb-b100-abad3ec29772-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pxsl9\" (UID: \"bde81c2f-1b0b-4adb-b100-abad3ec29772\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:35.248432 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.248376    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd87m\" (UniqueName: \"kubernetes.io/projected/bde81c2f-1b0b-4adb-b100-abad3ec29772-kube-api-access-vd87m\") pod \"kuadrant-console-plugin-66b5774985-pxsl9\" (UID: \"bde81c2f-1b0b-4adb-b100-abad3ec29772\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:35.349022 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.348995    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde81c2f-1b0b-4adb-b100-abad3ec29772-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pxsl9\" (UID: \"bde81c2f-1b0b-4adb-b100-abad3ec29772\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:35.349113 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.349027    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd87m\" (UniqueName: \"kubernetes.io/projected/bde81c2f-1b0b-4adb-b100-abad3ec29772-kube-api-access-vd87m\") pod \"kuadrant-console-plugin-66b5774985-pxsl9\" (UID: \"bde81c2f-1b0b-4adb-b100-abad3ec29772\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:35.349113 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.349105    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bde81c2f-1b0b-4adb-b100-abad3ec29772-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pxsl9\" (UID: \"bde81c2f-1b0b-4adb-b100-abad3ec29772\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:35.349185 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:52:35.349144    2558 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 17 05:52:35.349227 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:52:35.349214    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bde81c2f-1b0b-4adb-b100-abad3ec29772-plugin-serving-cert podName:bde81c2f-1b0b-4adb-b100-abad3ec29772 nodeName:}" failed. No retries permitted until 2026-07-17 05:52:35.849194266 +0000 UTC m=+678.291132818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/bde81c2f-1b0b-4adb-b100-abad3ec29772-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-pxsl9" (UID: "bde81c2f-1b0b-4adb-b100-abad3ec29772") : secret "plugin-serving-cert" not found
Jul 17 05:52:35.349751 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.349734    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bde81c2f-1b0b-4adb-b100-abad3ec29772-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pxsl9\" (UID: \"bde81c2f-1b0b-4adb-b100-abad3ec29772\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:35.360081 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.360054    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd87m\" (UniqueName: \"kubernetes.io/projected/bde81c2f-1b0b-4adb-b100-abad3ec29772-kube-api-access-vd87m\") pod \"kuadrant-console-plugin-66b5774985-pxsl9\" (UID: \"bde81c2f-1b0b-4adb-b100-abad3ec29772\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:35.404400 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.404384    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wpslm"
Jul 17 05:52:35.852669 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.852643    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde81c2f-1b0b-4adb-b100-abad3ec29772-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pxsl9\" (UID: \"bde81c2f-1b0b-4adb-b100-abad3ec29772\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:35.855171 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:35.855148    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde81c2f-1b0b-4adb-b100-abad3ec29772-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pxsl9\" (UID: \"bde81c2f-1b0b-4adb-b100-abad3ec29772\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:36.069281 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:36.069255    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"
Jul 17 05:52:36.196685 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:36.196661    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9"]
Jul 17 05:52:36.198890 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:52:36.198866    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbde81c2f_1b0b_4adb_b100_abad3ec29772.slice/crio-e8fe7d8c1f20b5cc2648fefae1516b0575d8cd07c7477d9ecf5021fb7b19deb5 WatchSource:0}: Error finding container e8fe7d8c1f20b5cc2648fefae1516b0575d8cd07c7477d9ecf5021fb7b19deb5: Status 404 returned error can't find the container with id e8fe7d8c1f20b5cc2648fefae1516b0575d8cd07c7477d9ecf5021fb7b19deb5
Jul 17 05:52:36.200646 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:36.200629    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:52:36.464043 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:36.463982    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9" event={"ID":"bde81c2f-1b0b-4adb-b100-abad3ec29772","Type":"ContainerStarted","Data":"e8fe7d8c1f20b5cc2648fefae1516b0575d8cd07c7477d9ecf5021fb7b19deb5"}
Jul 17 05:52:44.456734 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:44.456698    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:44.457241 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:44.456790    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-47p8w"
Jul 17 05:52:46.093216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.093180    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"]
Jul 17 05:52:46.116886 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.116859    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"]
Jul 17 05:52:46.117062 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.116964    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:52:46.243868 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.243829    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdd93507-6551-42f4-afd3-9de05f5aff87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rj8hv\" (UID: \"cdd93507-6551-42f4-afd3-9de05f5aff87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:52:46.244047 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.243961    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grznl\" (UniqueName: \"kubernetes.io/projected/cdd93507-6551-42f4-afd3-9de05f5aff87-kube-api-access-grznl\") pod \"kuadrant-operator-controller-manager-55467d9f7-rj8hv\" (UID: \"cdd93507-6551-42f4-afd3-9de05f5aff87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:52:46.344833 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.344755    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grznl\" (UniqueName: \"kubernetes.io/projected/cdd93507-6551-42f4-afd3-9de05f5aff87-kube-api-access-grznl\") pod \"kuadrant-operator-controller-manager-55467d9f7-rj8hv\" (UID: \"cdd93507-6551-42f4-afd3-9de05f5aff87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:52:46.345008 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.344888    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdd93507-6551-42f4-afd3-9de05f5aff87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rj8hv\" (UID: \"cdd93507-6551-42f4-afd3-9de05f5aff87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:52:46.345357 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.345328    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdd93507-6551-42f4-afd3-9de05f5aff87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rj8hv\" (UID: \"cdd93507-6551-42f4-afd3-9de05f5aff87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:52:46.354341 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.354316    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grznl\" (UniqueName: \"kubernetes.io/projected/cdd93507-6551-42f4-afd3-9de05f5aff87-kube-api-access-grznl\") pod \"kuadrant-operator-controller-manager-55467d9f7-rj8hv\" (UID: \"cdd93507-6551-42f4-afd3-9de05f5aff87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:52:46.430547 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.430519    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:52:46.442027 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.442003    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"]
Jul 17 05:52:46.446139 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.446085    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"]
Jul 17 05:52:46.454423 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.454400    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"]
Jul 17 05:52:46.454822 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.454639    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf" podUID="babed64b-c70c-44fe-96c5-f5aedfa52efc" containerName="manager" containerID="cri-o://c7675f9d37eb5676cfbc93fb863287c8956eb23bc2b4ccbeb10902e0a5856c4e" gracePeriod=2
Jul 17 05:52:46.471248 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.471194    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"]
Jul 17 05:52:46.473415 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.473385    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"]
Jul 17 05:52:46.474352 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.474329    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="babed64b-c70c-44fe-96c5-f5aedfa52efc" containerName="manager"
Jul 17 05:52:46.474442 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.474356    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="babed64b-c70c-44fe-96c5-f5aedfa52efc" containerName="manager"
Jul 17 05:52:46.474549 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.474534    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="babed64b-c70c-44fe-96c5-f5aedfa52efc" containerName="manager"
Jul 17 05:52:46.480577 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.480552    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:52:46.488104 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:52:46.486573    2558 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"babed64b-c70c-44fe-96c5-f5aedfa52efc\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-h27xf\": pods \"kuadrant-operator-controller-manager-8684c9ff49-h27xf\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:46.489555 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.489528    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"]
Jul 17 05:52:46.491614 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:52:46.491509    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h27xf\" is forbidden: User \"system:node:ip-10-0-140-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-96.ec2.internal' and this object" podUID="babed64b-c70c-44fe-96c5-f5aedfa52efc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:46.494817 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.494767    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"]
Jul 17 05:52:46.494938 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.494845    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"]
Jul 17 05:52:46.495025 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.494965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:52:46.495109 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.495039    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp" podUID="48c22755-698f-4474-84af-39fed41e1b32" containerName="manager" containerID="cri-o://55a385ba47c5b161ba4ae9d77d3212c59e75e84082950e96f7a59d11d2db7b06" gracePeriod=2
Jul 17 05:52:46.497312 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.497265    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"]
Jul 17 05:52:46.503580 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.503524    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"]
Jul 17 05:52:46.506164 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.506145    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm"]
Jul 17 05:52:46.506674 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.506656    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48c22755-698f-4474-84af-39fed41e1b32" containerName="manager"
Jul 17 05:52:46.506767 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.506677    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="48c22755-698f-4474-84af-39fed41e1b32" containerName="manager"
Jul 17 05:52:46.506767 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.506760    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="48c22755-698f-4474-84af-39fed41e1b32" containerName="manager"
Jul 17 05:52:46.511975 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.511956    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm"
Jul 17 05:52:46.515919 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:52:46.515784    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4qglp\" is forbidden: User \"system:node:ip-10-0-140-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-96.ec2.internal' and this object" podUID="48c22755-698f-4474-84af-39fed41e1b32" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"
Jul 17 05:52:46.522043 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.520892    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm"]
Jul 17 05:52:46.563259 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:52:46.563217    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h27xf\" is forbidden: User \"system:node:ip-10-0-140-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-96.ec2.internal' and this object" podUID="babed64b-c70c-44fe-96c5-f5aedfa52efc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:46.648123 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.648092    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20e0f3d2-5fd0-465f-af40-b450f247031b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-svcql\" (UID: \"20e0f3d2-5fd0-465f-af40-b450f247031b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:52:46.648256 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.648149    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1826652f-913c-4a35-affe-011ef9fa10c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rzzq2\" (UID: \"1826652f-913c-4a35-affe-011ef9fa10c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:52:46.648256 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.648177    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcbhk\" (UniqueName: \"kubernetes.io/projected/1826652f-913c-4a35-affe-011ef9fa10c9-kube-api-access-jcbhk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rzzq2\" (UID: \"1826652f-913c-4a35-affe-011ef9fa10c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:52:46.648256 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.648244    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pcw9\" (UniqueName: \"kubernetes.io/projected/e571449b-4eee-42e9-8042-193c482d1bb0-kube-api-access-5pcw9\") pod \"limitador-operator-controller-manager-859d7b78bd-5sdjm\" (UID: \"e571449b-4eee-42e9-8042-193c482d1bb0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm"
Jul 17 05:52:46.648387 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.648334    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v9hm\" (UniqueName: \"kubernetes.io/projected/20e0f3d2-5fd0-465f-af40-b450f247031b-kube-api-access-6v9hm\") pod \"kuadrant-operator-controller-manager-55467d9f7-svcql\" (UID: \"20e0f3d2-5fd0-465f-af40-b450f247031b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:52:46.749303 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.749271    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20e0f3d2-5fd0-465f-af40-b450f247031b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-svcql\" (UID: \"20e0f3d2-5fd0-465f-af40-b450f247031b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:52:46.749458 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.749350    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1826652f-913c-4a35-affe-011ef9fa10c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rzzq2\" (UID: \"1826652f-913c-4a35-affe-011ef9fa10c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:52:46.749458 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.749373    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcbhk\" (UniqueName: \"kubernetes.io/projected/1826652f-913c-4a35-affe-011ef9fa10c9-kube-api-access-jcbhk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rzzq2\" (UID: \"1826652f-913c-4a35-affe-011ef9fa10c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:52:46.749458 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.749395    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pcw9\" (UniqueName: \"kubernetes.io/projected/e571449b-4eee-42e9-8042-193c482d1bb0-kube-api-access-5pcw9\") pod \"limitador-operator-controller-manager-859d7b78bd-5sdjm\" (UID: \"e571449b-4eee-42e9-8042-193c482d1bb0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm"
Jul 17 05:52:46.749458 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.749425    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v9hm\" (UniqueName: \"kubernetes.io/projected/20e0f3d2-5fd0-465f-af40-b450f247031b-kube-api-access-6v9hm\") pod \"kuadrant-operator-controller-manager-55467d9f7-svcql\" (UID: \"20e0f3d2-5fd0-465f-af40-b450f247031b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:52:46.749749 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.749722    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20e0f3d2-5fd0-465f-af40-b450f247031b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-svcql\" (UID: \"20e0f3d2-5fd0-465f-af40-b450f247031b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:52:46.749918 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.749722    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1826652f-913c-4a35-affe-011ef9fa10c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rzzq2\" (UID: \"1826652f-913c-4a35-affe-011ef9fa10c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:52:46.762517 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.762456    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v9hm\" (UniqueName: \"kubernetes.io/projected/20e0f3d2-5fd0-465f-af40-b450f247031b-kube-api-access-6v9hm\") pod \"kuadrant-operator-controller-manager-55467d9f7-svcql\" (UID: \"20e0f3d2-5fd0-465f-af40-b450f247031b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:52:46.762993 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.762969    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pcw9\" (UniqueName: \"kubernetes.io/projected/e571449b-4eee-42e9-8042-193c482d1bb0-kube-api-access-5pcw9\") pod \"limitador-operator-controller-manager-859d7b78bd-5sdjm\" (UID: \"e571449b-4eee-42e9-8042-193c482d1bb0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm"
Jul 17 05:52:46.763152 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.763135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcbhk\" (UniqueName: \"kubernetes.io/projected/1826652f-913c-4a35-affe-011ef9fa10c9-kube-api-access-jcbhk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rzzq2\" (UID: \"1826652f-913c-4a35-affe-011ef9fa10c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:52:46.890898 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.890819    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:52:46.899957 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.899938    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:52:46.907466 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:46.907441    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm"
Jul 17 05:52:48.077687 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:52:48.077641    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4qglp\" is forbidden: User \"system:node:ip-10-0-140-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-96.ec2.internal' and this object" podUID="48c22755-698f-4474-84af-39fed41e1b32" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"
Jul 17 05:52:48.079748 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:52:48.079719    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h27xf\" is forbidden: User \"system:node:ip-10-0-140-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-96.ec2.internal' and this object" podUID="babed64b-c70c-44fe-96c5-f5aedfa52efc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:57.593959 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.593793    2558 generic.go:356] "Generic (PLEG): container finished" podID="48c22755-698f-4474-84af-39fed41e1b32" containerID="55a385ba47c5b161ba4ae9d77d3212c59e75e84082950e96f7a59d11d2db7b06" exitCode=0
Jul 17 05:52:57.603130 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.603016    2558 generic.go:356] "Generic (PLEG): container finished" podID="babed64b-c70c-44fe-96c5-f5aedfa52efc" containerID="c7675f9d37eb5676cfbc93fb863287c8956eb23bc2b4ccbeb10902e0a5856c4e" exitCode=0
Jul 17 05:52:57.702866 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.702758    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"]
Jul 17 05:52:57.707447 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:52:57.707413    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1826652f_913c_4a35_affe_011ef9fa10c9.slice/crio-4fff6836aa3e994fe1da0acdf23d29d3e599560e341bd94fcdc25dbbbcd00dcb WatchSource:0}: Error finding container 4fff6836aa3e994fe1da0acdf23d29d3e599560e341bd94fcdc25dbbbcd00dcb: Status 404 returned error can't find the container with id 4fff6836aa3e994fe1da0acdf23d29d3e599560e341bd94fcdc25dbbbcd00dcb
Jul 17 05:52:57.710311 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.710276    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:57.712855 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.712835    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"
Jul 17 05:52:57.724789 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.724720    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"]
Jul 17 05:52:57.728633 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:52:57.728609    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20e0f3d2_5fd0_465f_af40_b450f247031b.slice/crio-75b7b7af8c3a36b3f84fb4bb421231918b047b02500a41040fd3290b09a24a3e WatchSource:0}: Error finding container 75b7b7af8c3a36b3f84fb4bb421231918b047b02500a41040fd3290b09a24a3e: Status 404 returned error can't find the container with id 75b7b7af8c3a36b3f84fb4bb421231918b047b02500a41040fd3290b09a24a3e
Jul 17 05:52:57.743498 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.743457    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/babed64b-c70c-44fe-96c5-f5aedfa52efc-extensions-socket-volume\") pod \"babed64b-c70c-44fe-96c5-f5aedfa52efc\" (UID: \"babed64b-c70c-44fe-96c5-f5aedfa52efc\") "
Jul 17 05:52:57.743661 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.743646    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5bnl\" (UniqueName: \"kubernetes.io/projected/babed64b-c70c-44fe-96c5-f5aedfa52efc-kube-api-access-h5bnl\") pod \"babed64b-c70c-44fe-96c5-f5aedfa52efc\" (UID: \"babed64b-c70c-44fe-96c5-f5aedfa52efc\") "
Jul 17 05:52:57.743732 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.743694    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm2ck\" (UniqueName: \"kubernetes.io/projected/48c22755-698f-4474-84af-39fed41e1b32-kube-api-access-vm2ck\") pod \"48c22755-698f-4474-84af-39fed41e1b32\" (UID: \"48c22755-698f-4474-84af-39fed41e1b32\") "
Jul 17 05:52:57.743993 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.743961    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/babed64b-c70c-44fe-96c5-f5aedfa52efc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "babed64b-c70c-44fe-96c5-f5aedfa52efc" (UID: "babed64b-c70c-44fe-96c5-f5aedfa52efc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:52:57.745767 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.745743    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/babed64b-c70c-44fe-96c5-f5aedfa52efc-kube-api-access-h5bnl" (OuterVolumeSpecName: "kube-api-access-h5bnl") pod "babed64b-c70c-44fe-96c5-f5aedfa52efc" (UID: "babed64b-c70c-44fe-96c5-f5aedfa52efc"). InnerVolumeSpecName "kube-api-access-h5bnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:52:57.746168 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.746147    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c22755-698f-4474-84af-39fed41e1b32-kube-api-access-vm2ck" (OuterVolumeSpecName: "kube-api-access-vm2ck") pod "48c22755-698f-4474-84af-39fed41e1b32" (UID: "48c22755-698f-4474-84af-39fed41e1b32"). InnerVolumeSpecName "kube-api-access-vm2ck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:52:57.753827 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.753763    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm"]
Jul 17 05:52:57.755798 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:52:57.755664    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode571449b_4eee_42e9_8042_193c482d1bb0.slice/crio-58a8873d853790f3f388eae007d8f3bfd58bfc04c6bd6361dd5ca5df13cb43eb WatchSource:0}: Error finding container 58a8873d853790f3f388eae007d8f3bfd58bfc04c6bd6361dd5ca5df13cb43eb: Status 404 returned error can't find the container with id 58a8873d853790f3f388eae007d8f3bfd58bfc04c6bd6361dd5ca5df13cb43eb
Jul 17 05:52:57.844434 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.844405    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm2ck\" (UniqueName: \"kubernetes.io/projected/48c22755-698f-4474-84af-39fed41e1b32-kube-api-access-vm2ck\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:52:57.844434 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.844435    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/babed64b-c70c-44fe-96c5-f5aedfa52efc-extensions-socket-volume\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:52:57.844600 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:57.844445    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5bnl\" (UniqueName: \"kubernetes.io/projected/babed64b-c70c-44fe-96c5-f5aedfa52efc-kube-api-access-h5bnl\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:52:58.078122 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.078042    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48c22755-698f-4474-84af-39fed41e1b32" path="/var/lib/kubelet/pods/48c22755-698f-4474-84af-39fed41e1b32/volumes"
Jul 17 05:52:58.078392 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.078378    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="babed64b-c70c-44fe-96c5-f5aedfa52efc" path="/var/lib/kubelet/pods/babed64b-c70c-44fe-96c5-f5aedfa52efc/volumes"
Jul 17 05:52:58.609098 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.609057    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9" event={"ID":"bde81c2f-1b0b-4adb-b100-abad3ec29772","Type":"ContainerStarted","Data":"9885b741f99fdb6810acbf1c880bbed447c7dab9e9af042e74a189592708b824"}
Jul 17 05:52:58.610377 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.610357    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4qglp"
Jul 17 05:52:58.610524 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.610396    2558 scope.go:117] "RemoveContainer" containerID="55a385ba47c5b161ba4ae9d77d3212c59e75e84082950e96f7a59d11d2db7b06"
Jul 17 05:52:58.611963 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.611946    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h27xf"
Jul 17 05:52:58.613603 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.613578    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm" event={"ID":"e571449b-4eee-42e9-8042-193c482d1bb0","Type":"ContainerStarted","Data":"e42ee2a457690c08757db8c1af20278c07073af024860fef02f5d9d942d6875c"}
Jul 17 05:52:58.613723 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.613631    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm"
Jul 17 05:52:58.613723 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.613657    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm" event={"ID":"e571449b-4eee-42e9-8042-193c482d1bb0","Type":"ContainerStarted","Data":"58a8873d853790f3f388eae007d8f3bfd58bfc04c6bd6361dd5ca5df13cb43eb"}
Jul 17 05:52:58.615456 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.615430    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2" event={"ID":"1826652f-913c-4a35-affe-011ef9fa10c9","Type":"ContainerStarted","Data":"0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f"}
Jul 17 05:52:58.615577 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.615506    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:52:58.615577 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.615529    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2" event={"ID":"1826652f-913c-4a35-affe-011ef9fa10c9","Type":"ContainerStarted","Data":"4fff6836aa3e994fe1da0acdf23d29d3e599560e341bd94fcdc25dbbbcd00dcb"}
Jul 17 05:52:58.616973 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.616956    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql" event={"ID":"20e0f3d2-5fd0-465f-af40-b450f247031b","Type":"ContainerStarted","Data":"71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5"}
Jul 17 05:52:58.617061 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.616989    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql" event={"ID":"20e0f3d2-5fd0-465f-af40-b450f247031b","Type":"ContainerStarted","Data":"75b7b7af8c3a36b3f84fb4bb421231918b047b02500a41040fd3290b09a24a3e"}
Jul 17 05:52:58.617061 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.617012    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:52:58.621584 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.621567    2558 scope.go:117] "RemoveContainer" containerID="c7675f9d37eb5676cfbc93fb863287c8956eb23bc2b4ccbeb10902e0a5856c4e"
Jul 17 05:52:58.627710 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.627052    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pxsl9" podStartSLOduration=2.210209802 podStartE2EDuration="23.627034143s" totalImagesPullingTime="21.416824341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:52:58.625547448 +0000 UTC m=+701.067486023" watchObservedRunningTime="2026-07-17 05:52:58.627034143 +0000 UTC m=+701.068972717"
Jul 17 05:52:58.646042 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.646004    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql" podStartSLOduration=12.645993282 podStartE2EDuration="12.645993282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:52:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:52:58.644460777 +0000 UTC m=+701.086399390" watchObservedRunningTime="2026-07-17 05:52:58.645993282 +0000 UTC m=+701.087931856"
Jul 17 05:52:58.660681 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.660631    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm" podStartSLOduration=12.660618161 podStartE2EDuration="12.660618161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:52:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:52:58.659960921 +0000 UTC m=+701.101899498" watchObservedRunningTime="2026-07-17 05:52:58.660618161 +0000 UTC m=+701.102556792"
Jul 17 05:52:58.682004 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:52:58.681949    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2" podStartSLOduration=12.681932906 podStartE2EDuration="12.681932906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:52:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:52:58.677750881 +0000 UTC m=+701.119689455" watchObservedRunningTime="2026-07-17 05:52:58.681932906 +0000 UTC m=+701.123871480"
Jul 17 05:53:00.099644 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:53:00.099611    2558 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-rj8hv_kuadrant-system_cdd93507-6551-42f4-afd3-9de05f5aff87_0(ff91c8fab37009c31a87e0f898faf75f229d73e07f58413ebf1989fe6026ef19): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-rj8hv to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 17 05:53:00.099885 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:53:00.099666    2558 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-rj8hv_kuadrant-system_cdd93507-6551-42f4-afd3-9de05f5aff87_0(ff91c8fab37009c31a87e0f898faf75f229d73e07f58413ebf1989fe6026ef19): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-rj8hv to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:53:00.630381 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:00.630359    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:53:00.633366 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:53:00.633341    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-rj8hv\" is forbidden: User \"system:node:ip-10-0-140-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-96.ec2.internal' and this object" podUID="cdd93507-6551-42f4-afd3-9de05f5aff87" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:53:00.637215 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:00.637192    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:53:00.639454 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:53:00.639432    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-rj8hv\" is forbidden: User \"system:node:ip-10-0-140-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-96.ec2.internal' and this object" podUID="cdd93507-6551-42f4-afd3-9de05f5aff87" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:53:00.660659 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:00.660633    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grznl\" (UniqueName: \"kubernetes.io/projected/cdd93507-6551-42f4-afd3-9de05f5aff87-kube-api-access-grznl\") pod \"cdd93507-6551-42f4-afd3-9de05f5aff87\" (UID: \"cdd93507-6551-42f4-afd3-9de05f5aff87\") "
Jul 17 05:53:00.660759 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:00.660671    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdd93507-6551-42f4-afd3-9de05f5aff87-extensions-socket-volume\") pod \"cdd93507-6551-42f4-afd3-9de05f5aff87\" (UID: \"cdd93507-6551-42f4-afd3-9de05f5aff87\") "
Jul 17 05:53:00.660952 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:00.660927    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd93507-6551-42f4-afd3-9de05f5aff87-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cdd93507-6551-42f4-afd3-9de05f5aff87" (UID: "cdd93507-6551-42f4-afd3-9de05f5aff87"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:53:00.662723 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:00.662702    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd93507-6551-42f4-afd3-9de05f5aff87-kube-api-access-grznl" (OuterVolumeSpecName: "kube-api-access-grznl") pod "cdd93507-6551-42f4-afd3-9de05f5aff87" (UID: "cdd93507-6551-42f4-afd3-9de05f5aff87"). InnerVolumeSpecName "kube-api-access-grznl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:53:00.761796 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:00.761773    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grznl\" (UniqueName: \"kubernetes.io/projected/cdd93507-6551-42f4-afd3-9de05f5aff87-kube-api-access-grznl\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:53:00.761796 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:00.761794    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdd93507-6551-42f4-afd3-9de05f5aff87-extensions-socket-volume\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:53:01.635085 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:01.635051    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:53:01.637613 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:53:01.637583    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-rj8hv\" is forbidden: User \"system:node:ip-10-0-140-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-96.ec2.internal' and this object" podUID="cdd93507-6551-42f4-afd3-9de05f5aff87" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:53:01.642779 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:53:01.642750    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-rj8hv\" is forbidden: User \"system:node:ip-10-0-140-96.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-96.ec2.internal' and this object" podUID="cdd93507-6551-42f4-afd3-9de05f5aff87" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rj8hv"
Jul 17 05:53:02.078593 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:02.078526    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdd93507-6551-42f4-afd3-9de05f5aff87" path="/var/lib/kubelet/pods/cdd93507-6551-42f4-afd3-9de05f5aff87/volumes"
Jul 17 05:53:09.627188 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:09.627118    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:53:09.627188 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:09.627184    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5sdjm"
Jul 17 05:53:09.627669 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:09.627351    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:53:09.710943 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:09.710906    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"]
Jul 17 05:53:09.711216 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:09.711178    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2" podUID="1826652f-913c-4a35-affe-011ef9fa10c9" containerName="manager" containerID="cri-o://0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f" gracePeriod=10
Jul 17 05:53:09.941583 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:09.941557    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"]
Jul 17 05:53:09.944436 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:09.944421    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 05:53:09.956419 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:09.956394    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"]
Jul 17 05:53:09.965266 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:09.965242    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:53:10.029036 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.029011    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1826652f-913c-4a35-affe-011ef9fa10c9-extensions-socket-volume\") pod \"1826652f-913c-4a35-affe-011ef9fa10c9\" (UID: \"1826652f-913c-4a35-affe-011ef9fa10c9\") "
Jul 17 05:53:10.029156 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.029114    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcbhk\" (UniqueName: \"kubernetes.io/projected/1826652f-913c-4a35-affe-011ef9fa10c9-kube-api-access-jcbhk\") pod \"1826652f-913c-4a35-affe-011ef9fa10c9\" (UID: \"1826652f-913c-4a35-affe-011ef9fa10c9\") "
Jul 17 05:53:10.029254 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.029233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3214c6c9-d799-40c5-9acc-c3e294139f62-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p2wjf\" (UID: \"3214c6c9-d799-40c5-9acc-c3e294139f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 05:53:10.029305 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.029285    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4tvq\" (UniqueName: \"kubernetes.io/projected/3214c6c9-d799-40c5-9acc-c3e294139f62-kube-api-access-m4tvq\") pod \"kuadrant-operator-controller-manager-86878c769f-p2wjf\" (UID: \"3214c6c9-d799-40c5-9acc-c3e294139f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 05:53:10.029413 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.029388    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1826652f-913c-4a35-affe-011ef9fa10c9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1826652f-913c-4a35-affe-011ef9fa10c9" (UID: "1826652f-913c-4a35-affe-011ef9fa10c9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:53:10.029538 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.029501    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1826652f-913c-4a35-affe-011ef9fa10c9-extensions-socket-volume\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:53:10.031033 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.031007    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1826652f-913c-4a35-affe-011ef9fa10c9-kube-api-access-jcbhk" (OuterVolumeSpecName: "kube-api-access-jcbhk") pod "1826652f-913c-4a35-affe-011ef9fa10c9" (UID: "1826652f-913c-4a35-affe-011ef9fa10c9"). InnerVolumeSpecName "kube-api-access-jcbhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:53:10.130215 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.130143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3214c6c9-d799-40c5-9acc-c3e294139f62-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p2wjf\" (UID: \"3214c6c9-d799-40c5-9acc-c3e294139f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 05:53:10.130215 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.130180    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4tvq\" (UniqueName: \"kubernetes.io/projected/3214c6c9-d799-40c5-9acc-c3e294139f62-kube-api-access-m4tvq\") pod \"kuadrant-operator-controller-manager-86878c769f-p2wjf\" (UID: \"3214c6c9-d799-40c5-9acc-c3e294139f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 05:53:10.130408 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.130256    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcbhk\" (UniqueName: \"kubernetes.io/projected/1826652f-913c-4a35-affe-011ef9fa10c9-kube-api-access-jcbhk\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:53:10.130581 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.130557    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3214c6c9-d799-40c5-9acc-c3e294139f62-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p2wjf\" (UID: \"3214c6c9-d799-40c5-9acc-c3e294139f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 05:53:10.142265 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.142236    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4tvq\" (UniqueName: \"kubernetes.io/projected/3214c6c9-d799-40c5-9acc-c3e294139f62-kube-api-access-m4tvq\") pod \"kuadrant-operator-controller-manager-86878c769f-p2wjf\" (UID: \"3214c6c9-d799-40c5-9acc-c3e294139f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 05:53:10.272005 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.271966    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 05:53:10.441342 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.441316    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"]
Jul 17 05:53:10.443062 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:53:10.443028    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3214c6c9_d799_40c5_9acc_c3e294139f62.slice/crio-db8c14cc34df0914ab2d7288ddd29372410b6aa190e1e866a5488810d62df351 WatchSource:0}: Error finding container db8c14cc34df0914ab2d7288ddd29372410b6aa190e1e866a5488810d62df351: Status 404 returned error can't find the container with id db8c14cc34df0914ab2d7288ddd29372410b6aa190e1e866a5488810d62df351
Jul 17 05:53:10.688622 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.688591    2558 generic.go:356] "Generic (PLEG): container finished" podID="1826652f-913c-4a35-affe-011ef9fa10c9" containerID="0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f" exitCode=0
Jul 17 05:53:10.689022 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.688632    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2" event={"ID":"1826652f-913c-4a35-affe-011ef9fa10c9","Type":"ContainerDied","Data":"0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f"}
Jul 17 05:53:10.689022 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.688646    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"
Jul 17 05:53:10.689022 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.688687    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2" event={"ID":"1826652f-913c-4a35-affe-011ef9fa10c9","Type":"ContainerDied","Data":"4fff6836aa3e994fe1da0acdf23d29d3e599560e341bd94fcdc25dbbbcd00dcb"}
Jul 17 05:53:10.689022 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.688717    2558 scope.go:117] "RemoveContainer" containerID="0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f"
Jul 17 05:53:10.691008 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.690951    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf" event={"ID":"3214c6c9-d799-40c5-9acc-c3e294139f62","Type":"ContainerStarted","Data":"91588442de1696dafc473b9c3e41a997743887a8634af1199974bd499488abf9"}
Jul 17 05:53:10.691096 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.691005    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf" event={"ID":"3214c6c9-d799-40c5-9acc-c3e294139f62","Type":"ContainerStarted","Data":"db8c14cc34df0914ab2d7288ddd29372410b6aa190e1e866a5488810d62df351"}
Jul 17 05:53:10.691288 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.691271    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 05:53:10.697837 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.697822    2558 scope.go:117] "RemoveContainer" containerID="0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f"
Jul 17 05:53:10.698078 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:53:10.698059    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f\": container with ID starting with 0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f not found: ID does not exist" containerID="0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f"
Jul 17 05:53:10.698144 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.698090    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f"} err="failed to get container status \"0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f\": rpc error: code = NotFound desc = could not find container \"0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f\": container with ID starting with 0e3758cad30237200966c054ba52133e21026f9eb7498309740f2fc993310e3f not found: ID does not exist"
Jul 17 05:53:10.712174 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.712136    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf" podStartSLOduration=1.7121134630000001 podStartE2EDuration="1.712113463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:53:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:53:10.710532346 +0000 UTC m=+713.152470940" watchObservedRunningTime="2026-07-17 05:53:10.712113463 +0000 UTC m=+713.154052036"
Jul 17 05:53:10.724286 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.724260    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"]
Jul 17 05:53:10.727750 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:10.727727    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rzzq2"]
Jul 17 05:53:12.078044 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:12.078015    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1826652f-913c-4a35-affe-011ef9fa10c9" path="/var/lib/kubelet/pods/1826652f-913c-4a35-affe-011ef9fa10c9/volumes"
Jul 17 05:53:21.699559 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:21.699532    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 05:53:21.744422 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:21.744344    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"]
Jul 17 05:53:21.744887 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:21.744840    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql" podUID="20e0f3d2-5fd0-465f-af40-b450f247031b" containerName="manager" containerID="cri-o://71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5" gracePeriod=10
Jul 17 05:53:22.510439 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.510416    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:53:22.632158 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.632093    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20e0f3d2-5fd0-465f-af40-b450f247031b-extensions-socket-volume\") pod \"20e0f3d2-5fd0-465f-af40-b450f247031b\" (UID: \"20e0f3d2-5fd0-465f-af40-b450f247031b\") "
Jul 17 05:53:22.632158 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.632140    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v9hm\" (UniqueName: \"kubernetes.io/projected/20e0f3d2-5fd0-465f-af40-b450f247031b-kube-api-access-6v9hm\") pod \"20e0f3d2-5fd0-465f-af40-b450f247031b\" (UID: \"20e0f3d2-5fd0-465f-af40-b450f247031b\") "
Jul 17 05:53:22.632416 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.632388    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e0f3d2-5fd0-465f-af40-b450f247031b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "20e0f3d2-5fd0-465f-af40-b450f247031b" (UID: "20e0f3d2-5fd0-465f-af40-b450f247031b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:53:22.634167 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.634143    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20e0f3d2-5fd0-465f-af40-b450f247031b-kube-api-access-6v9hm" (OuterVolumeSpecName: "kube-api-access-6v9hm") pod "20e0f3d2-5fd0-465f-af40-b450f247031b" (UID: "20e0f3d2-5fd0-465f-af40-b450f247031b"). InnerVolumeSpecName "kube-api-access-6v9hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:53:22.732768 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.732742    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20e0f3d2-5fd0-465f-af40-b450f247031b-extensions-socket-volume\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:53:22.732768 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.732767    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v9hm\" (UniqueName: \"kubernetes.io/projected/20e0f3d2-5fd0-465f-af40-b450f247031b-kube-api-access-6v9hm\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:53:22.752119 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.752094    2558 generic.go:356] "Generic (PLEG): container finished" podID="20e0f3d2-5fd0-465f-af40-b450f247031b" containerID="71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5" exitCode=0
Jul 17 05:53:22.752203 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.752141    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"
Jul 17 05:53:22.752203 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.752170    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql" event={"ID":"20e0f3d2-5fd0-465f-af40-b450f247031b","Type":"ContainerDied","Data":"71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5"}
Jul 17 05:53:22.752283 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.752213    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql" event={"ID":"20e0f3d2-5fd0-465f-af40-b450f247031b","Type":"ContainerDied","Data":"75b7b7af8c3a36b3f84fb4bb421231918b047b02500a41040fd3290b09a24a3e"}
Jul 17 05:53:22.752283 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.752235    2558 scope.go:117] "RemoveContainer" containerID="71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5"
Jul 17 05:53:22.766647 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.766633    2558 scope.go:117] "RemoveContainer" containerID="71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5"
Jul 17 05:53:22.766910 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:53:22.766892    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5\": container with ID starting with 71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5 not found: ID does not exist" containerID="71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5"
Jul 17 05:53:22.766960 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.766916    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5"} err="failed to get container status \"71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5\": rpc error: code = NotFound desc = could not find container \"71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5\": container with ID starting with 71b222e871dcdd18465d826b712d4cf6deba6fbd9d95c491c5b4686559763dd5 not found: ID does not exist"
Jul 17 05:53:22.774975 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.774954    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"]
Jul 17 05:53:22.778823 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:22.778803    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svcql"]
Jul 17 05:53:24.078153 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:24.078114    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20e0f3d2-5fd0-465f-af40-b450f247031b" path="/var/lib/kubelet/pods/20e0f3d2-5fd0-465f-af40-b450f247031b/volumes"
Jul 17 05:53:26.042388 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.042357    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"]
Jul 17 05:53:26.042942 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.042922    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e0f3d2-5fd0-465f-af40-b450f247031b" containerName="manager"
Jul 17 05:53:26.043021 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.042947    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e0f3d2-5fd0-465f-af40-b450f247031b" containerName="manager"
Jul 17 05:53:26.043021 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.042966    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1826652f-913c-4a35-affe-011ef9fa10c9" containerName="manager"
Jul 17 05:53:26.043021 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.042974    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1826652f-913c-4a35-affe-011ef9fa10c9" containerName="manager"
Jul 17 05:53:26.043161 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.043073    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e0f3d2-5fd0-465f-af40-b450f247031b" containerName="manager"
Jul 17 05:53:26.043161 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.043088    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1826652f-913c-4a35-affe-011ef9fa10c9" containerName="manager"
Jul 17 05:53:26.112567 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.112542    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.115100 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.115081    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5q5bt\""
Jul 17 05:53:26.119906 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.119584    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"]
Jul 17 05:53:26.157688 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.157661    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.157825 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.157704    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gc7q\" (UniqueName: \"kubernetes.io/projected/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-kube-api-access-2gc7q\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.157825 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.157738    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.157939 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.157817    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.157939 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.157861    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.157939 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.157891    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.158052 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.157972    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.158052 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.158006    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.158052 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.158029    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.258728 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.258696    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.258728 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.258730    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.258907 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.258747    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.258907 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.258788    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.258907 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.258810    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.258907 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.258830    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.258907 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.258859    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.258907 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.258897    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gc7q\" (UniqueName: \"kubernetes.io/projected/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-kube-api-access-2gc7q\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.259199 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.258925    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.259199 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.259145    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.259312 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.259227    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.259365 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.259316    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.259408 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.259374    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.259694 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.259676    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.261754 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.261731    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.261913 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.261825    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.268104 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.268079    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gc7q\" (UniqueName: \"kubernetes.io/projected/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-kube-api-access-2gc7q\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.268319 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.268300    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc064b1-bea1-41de-9e44-b10de3e4c1b3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-g2wg5\" (UID: \"8cc064b1-bea1-41de-9e44-b10de3e4c1b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.423060 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.423040    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:26.601763 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.601740    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"]
Jul 17 05:53:26.604236 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:53:26.604207    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc064b1_bea1_41de_9e44_b10de3e4c1b3.slice/crio-0d7277a6ddf1c2984b9d4c0104c8f60e1817a7ce160fdd331fc0ebdcdfb923d7 WatchSource:0}: Error finding container 0d7277a6ddf1c2984b9d4c0104c8f60e1817a7ce160fdd331fc0ebdcdfb923d7: Status 404 returned error can't find the container with id 0d7277a6ddf1c2984b9d4c0104c8f60e1817a7ce160fdd331fc0ebdcdfb923d7
Jul 17 05:53:26.606335 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.606303    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 05:53:26.606429 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.606372    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 05:53:26.606429 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.606398    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 05:53:26.777849 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.777817    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5" event={"ID":"8cc064b1-bea1-41de-9e44-b10de3e4c1b3","Type":"ContainerStarted","Data":"a9370e0d88f0b24ab2b1d82f34348146c2a9dd9ee225ca7b7aedfb92a539144e"}
Jul 17 05:53:26.777986 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.777877    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5" event={"ID":"8cc064b1-bea1-41de-9e44-b10de3e4c1b3","Type":"ContainerStarted","Data":"0d7277a6ddf1c2984b9d4c0104c8f60e1817a7ce160fdd331fc0ebdcdfb923d7"}
Jul 17 05:53:26.798141 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:26.798080    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5" podStartSLOduration=0.798058929 podStartE2EDuration="798.058929ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:53:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:53:26.795407079 +0000 UTC m=+729.237345663" watchObservedRunningTime="2026-07-17 05:53:26.798058929 +0000 UTC m=+729.239997504"
Jul 17 05:53:27.424164 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:27.424131    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:27.428971 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:27.428945    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:27.782791 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:27.782719    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:27.783521 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:27.783501    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:53:42.755953 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:42.755916    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-gggxn"]
Jul 17 05:53:42.758511 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:42.758490    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gggxn"
Jul 17 05:53:42.761347 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:42.761326    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5b7zn\""
Jul 17 05:53:42.766339 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:42.766318    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gggxn"]
Jul 17 05:53:42.884001 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:42.883977    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcq7r\" (UniqueName: \"kubernetes.io/projected/4bdd30de-733f-45fa-8e31-a370bef8c45d-kube-api-access-lcq7r\") pod \"authorino-58cd955675-gggxn\" (UID: \"4bdd30de-733f-45fa-8e31-a370bef8c45d\") " pod="kuadrant-system/authorino-58cd955675-gggxn"
Jul 17 05:53:42.985176 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:42.985150    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcq7r\" (UniqueName: \"kubernetes.io/projected/4bdd30de-733f-45fa-8e31-a370bef8c45d-kube-api-access-lcq7r\") pod \"authorino-58cd955675-gggxn\" (UID: \"4bdd30de-733f-45fa-8e31-a370bef8c45d\") " pod="kuadrant-system/authorino-58cd955675-gggxn"
Jul 17 05:53:42.993772 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:42.993745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcq7r\" (UniqueName: \"kubernetes.io/projected/4bdd30de-733f-45fa-8e31-a370bef8c45d-kube-api-access-lcq7r\") pod \"authorino-58cd955675-gggxn\" (UID: \"4bdd30de-733f-45fa-8e31-a370bef8c45d\") " pod="kuadrant-system/authorino-58cd955675-gggxn"
Jul 17 05:53:43.068780 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:43.068725    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gggxn"
Jul 17 05:53:43.196450 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:43.196422    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gggxn"]
Jul 17 05:53:43.199624 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:53:43.199591    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bdd30de_733f_45fa_8e31_a370bef8c45d.slice/crio-e151173a8788ec4625911c4af8be23eeeab29737dc14034a4ccb4e440e8b24d2 WatchSource:0}: Error finding container e151173a8788ec4625911c4af8be23eeeab29737dc14034a4ccb4e440e8b24d2: Status 404 returned error can't find the container with id e151173a8788ec4625911c4af8be23eeeab29737dc14034a4ccb4e440e8b24d2
Jul 17 05:53:43.861522 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:43.861462    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gggxn" event={"ID":"4bdd30de-733f-45fa-8e31-a370bef8c45d","Type":"ContainerStarted","Data":"e151173a8788ec4625911c4af8be23eeeab29737dc14034a4ccb4e440e8b24d2"}
Jul 17 05:53:46.878745 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:46.878710    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gggxn" event={"ID":"4bdd30de-733f-45fa-8e31-a370bef8c45d","Type":"ContainerStarted","Data":"21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48"}
Jul 17 05:53:46.893696 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:46.893649    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-gggxn" podStartSLOduration=2.028630526 podStartE2EDuration="4.89363645s" totalImagesPullingTime="2.865005924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:53:46.893218525 +0000 UTC m=+749.335157095" watchObservedRunningTime="2026-07-17 05:53:46.89363645 +0000 UTC m=+749.335575025"
Jul 17 05:53:47.454789 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:47.454760    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gggxn"]
Jul 17 05:53:48.887639 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:48.887597    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-gggxn" podUID="4bdd30de-733f-45fa-8e31-a370bef8c45d" containerName="authorino" containerID="cri-o://21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48" gracePeriod=30
Jul 17 05:53:49.132625 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.132603    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gggxn"
Jul 17 05:53:49.242091 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.242032    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcq7r\" (UniqueName: \"kubernetes.io/projected/4bdd30de-733f-45fa-8e31-a370bef8c45d-kube-api-access-lcq7r\") pod \"4bdd30de-733f-45fa-8e31-a370bef8c45d\" (UID: \"4bdd30de-733f-45fa-8e31-a370bef8c45d\") "
Jul 17 05:53:49.243974 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.243953    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bdd30de-733f-45fa-8e31-a370bef8c45d-kube-api-access-lcq7r" (OuterVolumeSpecName: "kube-api-access-lcq7r") pod "4bdd30de-733f-45fa-8e31-a370bef8c45d" (UID: "4bdd30de-733f-45fa-8e31-a370bef8c45d"). InnerVolumeSpecName "kube-api-access-lcq7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:53:49.343539 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.343509    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcq7r\" (UniqueName: \"kubernetes.io/projected/4bdd30de-733f-45fa-8e31-a370bef8c45d-kube-api-access-lcq7r\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:53:49.892999 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.892963    2558 generic.go:356] "Generic (PLEG): container finished" podID="4bdd30de-733f-45fa-8e31-a370bef8c45d" containerID="21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48" exitCode=0
Jul 17 05:53:49.893435 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.893011    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gggxn"
Jul 17 05:53:49.893435 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.893048    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gggxn" event={"ID":"4bdd30de-733f-45fa-8e31-a370bef8c45d","Type":"ContainerDied","Data":"21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48"}
Jul 17 05:53:49.893435 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.893088    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gggxn" event={"ID":"4bdd30de-733f-45fa-8e31-a370bef8c45d","Type":"ContainerDied","Data":"e151173a8788ec4625911c4af8be23eeeab29737dc14034a4ccb4e440e8b24d2"}
Jul 17 05:53:49.893435 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.893106    2558 scope.go:117] "RemoveContainer" containerID="21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48"
Jul 17 05:53:49.901855 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.901840    2558 scope.go:117] "RemoveContainer" containerID="21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48"
Jul 17 05:53:49.902105 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:53:49.902088    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48\": container with ID starting with 21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48 not found: ID does not exist" containerID="21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48"
Jul 17 05:53:49.902143 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.902114    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48"} err="failed to get container status \"21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48\": rpc error: code = NotFound desc = could not find container \"21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48\": container with ID starting with 21a236c1675652ef01a76a7fc5b40dd00fdbfdef292bf12956c28dc894dede48 not found: ID does not exist"
Jul 17 05:53:49.915400 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.915376    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gggxn"]
Jul 17 05:53:49.919012 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:49.918992    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-gggxn"]
Jul 17 05:53:50.077822 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:53:50.077794    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bdd30de-733f-45fa-8e31-a370bef8c45d" path="/var/lib/kubelet/pods/4bdd30de-733f-45fa-8e31-a370bef8c45d/volumes"
Jul 17 05:54:12.092061 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.092031    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mx8j7"]
Jul 17 05:54:12.092454 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.092393    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bdd30de-733f-45fa-8e31-a370bef8c45d" containerName="authorino"
Jul 17 05:54:12.092454 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.092407    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bdd30de-733f-45fa-8e31-a370bef8c45d" containerName="authorino"
Jul 17 05:54:12.092560 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.092503    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bdd30de-733f-45fa-8e31-a370bef8c45d" containerName="authorino"
Jul 17 05:54:12.094600 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.094584    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mx8j7"
Jul 17 05:54:12.098466 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.098445    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5b7zn\""
Jul 17 05:54:12.110308 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.110286    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mx8j7"]
Jul 17 05:54:12.216237 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.216205    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lmh7\" (UniqueName: \"kubernetes.io/projected/9c09b575-754c-40c9-94ab-d29d587d8863-kube-api-access-7lmh7\") pod \"authorino-5d4b67c446-mx8j7\" (UID: \"9c09b575-754c-40c9-94ab-d29d587d8863\") " pod="kuadrant-system/authorino-5d4b67c446-mx8j7"
Jul 17 05:54:12.317608 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.317587    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lmh7\" (UniqueName: \"kubernetes.io/projected/9c09b575-754c-40c9-94ab-d29d587d8863-kube-api-access-7lmh7\") pod \"authorino-5d4b67c446-mx8j7\" (UID: \"9c09b575-754c-40c9-94ab-d29d587d8863\") " pod="kuadrant-system/authorino-5d4b67c446-mx8j7"
Jul 17 05:54:12.325266 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.325237    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lmh7\" (UniqueName: \"kubernetes.io/projected/9c09b575-754c-40c9-94ab-d29d587d8863-kube-api-access-7lmh7\") pod \"authorino-5d4b67c446-mx8j7\" (UID: \"9c09b575-754c-40c9-94ab-d29d587d8863\") " pod="kuadrant-system/authorino-5d4b67c446-mx8j7"
Jul 17 05:54:12.334564 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.334534    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mx8j7"]
Jul 17 05:54:12.334765 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.334750    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mx8j7"
Jul 17 05:54:12.356013 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.355987    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-677f78fcff-262ck"]
Jul 17 05:54:12.358785 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.358766    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677f78fcff-262ck"
Jul 17 05:54:12.367347 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.367313    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-677f78fcff-262ck"]
Jul 17 05:54:12.465615 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.465590    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mx8j7"]
Jul 17 05:54:12.468221 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:54:12.468192    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c09b575_754c_40c9_94ab_d29d587d8863.slice/crio-ebe3f6ea64cdf4f9e580c18d172bc497eb535f82ff2192646d9fd036970f48eb WatchSource:0}: Error finding container ebe3f6ea64cdf4f9e580c18d172bc497eb535f82ff2192646d9fd036970f48eb: Status 404 returned error can't find the container with id ebe3f6ea64cdf4f9e580c18d172bc497eb535f82ff2192646d9fd036970f48eb
Jul 17 05:54:12.468329 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.468269    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-677f78fcff-262ck"]
Jul 17 05:54:12.468556 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:54:12.468537    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9s6bt], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-677f78fcff-262ck" podUID="679fe53d-4e31-4501-8b39-231a4a62d90b"
Jul 17 05:54:12.491740 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.491712    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-749f4c9f87-4mdml"]
Jul 17 05:54:12.494323 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.494308    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-749f4c9f87-4mdml"
Jul 17 05:54:12.497184 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.497163    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 17 05:54:12.504025 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.504003    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-749f4c9f87-4mdml"]
Jul 17 05:54:12.519515 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.519490    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s6bt\" (UniqueName: \"kubernetes.io/projected/679fe53d-4e31-4501-8b39-231a4a62d90b-kube-api-access-9s6bt\") pod \"authorino-677f78fcff-262ck\" (UID: \"679fe53d-4e31-4501-8b39-231a4a62d90b\") " pod="kuadrant-system/authorino-677f78fcff-262ck"
Jul 17 05:54:12.620403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.620351    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s6bt\" (UniqueName: \"kubernetes.io/projected/679fe53d-4e31-4501-8b39-231a4a62d90b-kube-api-access-9s6bt\") pod \"authorino-677f78fcff-262ck\" (UID: \"679fe53d-4e31-4501-8b39-231a4a62d90b\") " pod="kuadrant-system/authorino-677f78fcff-262ck"
Jul 17 05:54:12.620403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.620387    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmgvg\" (UniqueName: \"kubernetes.io/projected/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-kube-api-access-qmgvg\") pod \"authorino-749f4c9f87-4mdml\" (UID: \"30ed9ccc-997f-411d-9e85-7c8d2230d6a1\") " pod="kuadrant-system/authorino-749f4c9f87-4mdml"
Jul 17 05:54:12.620585 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.620408    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-tls-cert\") pod \"authorino-749f4c9f87-4mdml\" (UID: \"30ed9ccc-997f-411d-9e85-7c8d2230d6a1\") " pod="kuadrant-system/authorino-749f4c9f87-4mdml"
Jul 17 05:54:12.628260 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.628230    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s6bt\" (UniqueName: \"kubernetes.io/projected/679fe53d-4e31-4501-8b39-231a4a62d90b-kube-api-access-9s6bt\") pod \"authorino-677f78fcff-262ck\" (UID: \"679fe53d-4e31-4501-8b39-231a4a62d90b\") " pod="kuadrant-system/authorino-677f78fcff-262ck"
Jul 17 05:54:12.721363 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.721339    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmgvg\" (UniqueName: \"kubernetes.io/projected/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-kube-api-access-qmgvg\") pod \"authorino-749f4c9f87-4mdml\" (UID: \"30ed9ccc-997f-411d-9e85-7c8d2230d6a1\") " pod="kuadrant-system/authorino-749f4c9f87-4mdml"
Jul 17 05:54:12.721445 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.721369    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-tls-cert\") pod \"authorino-749f4c9f87-4mdml\" (UID: \"30ed9ccc-997f-411d-9e85-7c8d2230d6a1\") " pod="kuadrant-system/authorino-749f4c9f87-4mdml"
Jul 17 05:54:12.723661 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.723643    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-tls-cert\") pod \"authorino-749f4c9f87-4mdml\" (UID: \"30ed9ccc-997f-411d-9e85-7c8d2230d6a1\") " pod="kuadrant-system/authorino-749f4c9f87-4mdml"
Jul 17 05:54:12.729011 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.728989    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmgvg\" (UniqueName: \"kubernetes.io/projected/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-kube-api-access-qmgvg\") pod \"authorino-749f4c9f87-4mdml\" (UID: \"30ed9ccc-997f-411d-9e85-7c8d2230d6a1\") " pod="kuadrant-system/authorino-749f4c9f87-4mdml"
Jul 17 05:54:12.804271 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.804251    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-749f4c9f87-4mdml"
Jul 17 05:54:12.935352 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:12.935329    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-749f4c9f87-4mdml"]
Jul 17 05:54:12.937032 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:54:12.937001    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30ed9ccc_997f_411d_9e85_7c8d2230d6a1.slice/crio-980a3215396cb41c7d2c12f5cdc6962a7b455755454281cbe720cfb630fe3dc2 WatchSource:0}: Error finding container 980a3215396cb41c7d2c12f5cdc6962a7b455755454281cbe720cfb630fe3dc2: Status 404 returned error can't find the container with id 980a3215396cb41c7d2c12f5cdc6962a7b455755454281cbe720cfb630fe3dc2
Jul 17 05:54:13.007849 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.007815    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mx8j7" event={"ID":"9c09b575-754c-40c9-94ab-d29d587d8863","Type":"ContainerStarted","Data":"0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e"}
Jul 17 05:54:13.007989 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.007867    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mx8j7" event={"ID":"9c09b575-754c-40c9-94ab-d29d587d8863","Type":"ContainerStarted","Data":"ebe3f6ea64cdf4f9e580c18d172bc497eb535f82ff2192646d9fd036970f48eb"}
Jul 17 05:54:13.007989 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.007860    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-mx8j7" podUID="9c09b575-754c-40c9-94ab-d29d587d8863" containerName="authorino" containerID="cri-o://0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e" gracePeriod=30
Jul 17 05:54:13.009354 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.009337    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677f78fcff-262ck"
Jul 17 05:54:13.009354 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.009347    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-749f4c9f87-4mdml" event={"ID":"30ed9ccc-997f-411d-9e85-7c8d2230d6a1","Type":"ContainerStarted","Data":"980a3215396cb41c7d2c12f5cdc6962a7b455755454281cbe720cfb630fe3dc2"}
Jul 17 05:54:13.014876 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.014857    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677f78fcff-262ck"
Jul 17 05:54:13.022541 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.022500    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-mx8j7" podStartSLOduration=0.569403938 podStartE2EDuration="1.02246665s" totalImagesPullingTime="453.062712ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:54:13.021493785 +0000 UTC m=+775.463432349" watchObservedRunningTime="2026-07-17 05:54:13.02246665 +0000 UTC m=+775.464405224"
Jul 17 05:54:13.123833 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.123812    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9s6bt\" (UniqueName: \"kubernetes.io/projected/679fe53d-4e31-4501-8b39-231a4a62d90b-kube-api-access-9s6bt\") pod \"679fe53d-4e31-4501-8b39-231a4a62d90b\" (UID: \"679fe53d-4e31-4501-8b39-231a4a62d90b\") "
Jul 17 05:54:13.125736 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.125710    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/679fe53d-4e31-4501-8b39-231a4a62d90b-kube-api-access-9s6bt" (OuterVolumeSpecName: "kube-api-access-9s6bt") pod "679fe53d-4e31-4501-8b39-231a4a62d90b" (UID: "679fe53d-4e31-4501-8b39-231a4a62d90b"). InnerVolumeSpecName "kube-api-access-9s6bt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:54:13.225069 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.225034    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s6bt\" (UniqueName: \"kubernetes.io/projected/679fe53d-4e31-4501-8b39-231a4a62d90b-kube-api-access-9s6bt\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:54:13.247955 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.247934    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mx8j7"
Jul 17 05:54:13.426980 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.426910    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lmh7\" (UniqueName: \"kubernetes.io/projected/9c09b575-754c-40c9-94ab-d29d587d8863-kube-api-access-7lmh7\") pod \"9c09b575-754c-40c9-94ab-d29d587d8863\" (UID: \"9c09b575-754c-40c9-94ab-d29d587d8863\") "
Jul 17 05:54:13.429182 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.429150    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c09b575-754c-40c9-94ab-d29d587d8863-kube-api-access-7lmh7" (OuterVolumeSpecName: "kube-api-access-7lmh7") pod "9c09b575-754c-40c9-94ab-d29d587d8863" (UID: "9c09b575-754c-40c9-94ab-d29d587d8863"). InnerVolumeSpecName "kube-api-access-7lmh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:54:13.527491 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:13.527445    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lmh7\" (UniqueName: \"kubernetes.io/projected/9c09b575-754c-40c9-94ab-d29d587d8863-kube-api-access-7lmh7\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:54:14.015458 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.015425    2558 generic.go:356] "Generic (PLEG): container finished" podID="9c09b575-754c-40c9-94ab-d29d587d8863" containerID="0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e" exitCode=0
Jul 17 05:54:14.015625 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.015488    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mx8j7"
Jul 17 05:54:14.015625 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.015514    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mx8j7" event={"ID":"9c09b575-754c-40c9-94ab-d29d587d8863","Type":"ContainerDied","Data":"0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e"}
Jul 17 05:54:14.015625 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.015556    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mx8j7" event={"ID":"9c09b575-754c-40c9-94ab-d29d587d8863","Type":"ContainerDied","Data":"ebe3f6ea64cdf4f9e580c18d172bc497eb535f82ff2192646d9fd036970f48eb"}
Jul 17 05:54:14.015625 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.015573    2558 scope.go:117] "RemoveContainer" containerID="0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e"
Jul 17 05:54:14.017652 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.017618    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-749f4c9f87-4mdml" event={"ID":"30ed9ccc-997f-411d-9e85-7c8d2230d6a1","Type":"ContainerStarted","Data":"7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78"}
Jul 17 05:54:14.017761 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.017670    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677f78fcff-262ck"
Jul 17 05:54:14.025415 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.025399    2558 scope.go:117] "RemoveContainer" containerID="0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e"
Jul 17 05:54:14.025694 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:54:14.025675    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e\": container with ID starting with 0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e not found: ID does not exist" containerID="0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e"
Jul 17 05:54:14.025772 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.025704    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e"} err="failed to get container status \"0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e\": rpc error: code = NotFound desc = could not find container \"0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e\": container with ID starting with 0fd2fb926e17ab5e1b1bea7dbe75c6cf840bbe6988f430033d69dbc01fc5117e not found: ID does not exist"
Jul 17 05:54:14.037208 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.037173    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-749f4c9f87-4mdml" podStartSLOduration=1.660438112 podStartE2EDuration="2.037161198s" totalImagesPullingTime="376.723086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:54:14.036171352 +0000 UTC m=+776.478109929" watchObservedRunningTime="2026-07-17 05:54:14.037161198 +0000 UTC m=+776.479099770"
Jul 17 05:54:14.051791 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.051764    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mx8j7"]
Jul 17 05:54:14.053525 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.053500    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mx8j7"]
Jul 17 05:54:14.079705 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.079674    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c09b575-754c-40c9-94ab-d29d587d8863" path="/var/lib/kubelet/pods/9c09b575-754c-40c9-94ab-d29d587d8863/volumes"
Jul 17 05:54:14.079960 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.079940    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-677f78fcff-262ck"]
Jul 17 05:54:14.088187 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:14.088165    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-677f78fcff-262ck"]
Jul 17 05:54:16.078124 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:16.078085    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="679fe53d-4e31-4501-8b39-231a4a62d90b" path="/var/lib/kubelet/pods/679fe53d-4e31-4501-8b39-231a4a62d90b/volumes"
Jul 17 05:54:18.754819 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.754790    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c6f6dcc69-w8vw4"]
Jul 17 05:54:18.755196 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.755163    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c09b575-754c-40c9-94ab-d29d587d8863" containerName="authorino"
Jul 17 05:54:18.755196 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.755174    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c09b575-754c-40c9-94ab-d29d587d8863" containerName="authorino"
Jul 17 05:54:18.755302 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.755248    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c09b575-754c-40c9-94ab-d29d587d8863" containerName="authorino"
Jul 17 05:54:18.759043 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.759024    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:18.763075 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.763048    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d5e01e9-aa7a-439c-a106-d616e12201a2-webhook-cert\") pod \"maas-controller-c6f6dcc69-w8vw4\" (UID: \"5d5e01e9-aa7a-439c-a106-d616e12201a2\") " pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:18.763196 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.763084    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjbvh\" (UniqueName: \"kubernetes.io/projected/5d5e01e9-aa7a-439c-a106-d616e12201a2-kube-api-access-vjbvh\") pod \"maas-controller-c6f6dcc69-w8vw4\" (UID: \"5d5e01e9-aa7a-439c-a106-d616e12201a2\") " pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:18.763570 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.763551    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 05:54:18.764076 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.764064    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9ncjx\""
Jul 17 05:54:18.764123 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.764088    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 05:54:18.772426 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.772401    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c6f6dcc69-w8vw4"]
Jul 17 05:54:18.863561 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.863532    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d5e01e9-aa7a-439c-a106-d616e12201a2-webhook-cert\") pod \"maas-controller-c6f6dcc69-w8vw4\" (UID: \"5d5e01e9-aa7a-439c-a106-d616e12201a2\") " pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:18.863561 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.863564    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjbvh\" (UniqueName: \"kubernetes.io/projected/5d5e01e9-aa7a-439c-a106-d616e12201a2-kube-api-access-vjbvh\") pod \"maas-controller-c6f6dcc69-w8vw4\" (UID: \"5d5e01e9-aa7a-439c-a106-d616e12201a2\") " pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:18.863757 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:54:18.863655    2558 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 17 05:54:18.863757 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:54:18.863712    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d5e01e9-aa7a-439c-a106-d616e12201a2-webhook-cert podName:5d5e01e9-aa7a-439c-a106-d616e12201a2 nodeName:}" failed. No retries permitted until 2026-07-17 05:54:19.363695119 +0000 UTC m=+781.805633671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/5d5e01e9-aa7a-439c-a106-d616e12201a2-webhook-cert") pod "maas-controller-c6f6dcc69-w8vw4" (UID: "5d5e01e9-aa7a-439c-a106-d616e12201a2") : secret "maas-controller-webhook-cert" not found
Jul 17 05:54:18.872231 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.872203    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjbvh\" (UniqueName: \"kubernetes.io/projected/5d5e01e9-aa7a-439c-a106-d616e12201a2-kube-api-access-vjbvh\") pod \"maas-controller-c6f6dcc69-w8vw4\" (UID: \"5d5e01e9-aa7a-439c-a106-d616e12201a2\") " pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:18.936123 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.936096    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64fd4654b6-bsnch"]
Jul 17 05:54:18.938877 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.938863    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:54:18.947695 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.947674    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-bsnch"]
Jul 17 05:54:18.964255 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.964218    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fgcw\" (UniqueName: \"kubernetes.io/projected/48a96c8d-490c-4283-8fcf-9c8e27cb3345-kube-api-access-5fgcw\") pod \"maas-controller-64fd4654b6-bsnch\" (UID: \"48a96c8d-490c-4283-8fcf-9c8e27cb3345\") " pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:54:18.964361 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:18.964285    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48a96c8d-490c-4283-8fcf-9c8e27cb3345-webhook-cert\") pod \"maas-controller-64fd4654b6-bsnch\" (UID: \"48a96c8d-490c-4283-8fcf-9c8e27cb3345\") " pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:54:19.065236 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:19.065172    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fgcw\" (UniqueName: \"kubernetes.io/projected/48a96c8d-490c-4283-8fcf-9c8e27cb3345-kube-api-access-5fgcw\") pod \"maas-controller-64fd4654b6-bsnch\" (UID: \"48a96c8d-490c-4283-8fcf-9c8e27cb3345\") " pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:54:19.065236 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:19.065213    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48a96c8d-490c-4283-8fcf-9c8e27cb3345-webhook-cert\") pod \"maas-controller-64fd4654b6-bsnch\" (UID: \"48a96c8d-490c-4283-8fcf-9c8e27cb3345\") " pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:54:19.067735 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:19.067707    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48a96c8d-490c-4283-8fcf-9c8e27cb3345-webhook-cert\") pod \"maas-controller-64fd4654b6-bsnch\" (UID: \"48a96c8d-490c-4283-8fcf-9c8e27cb3345\") " pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:54:19.073101 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:19.073076    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fgcw\" (UniqueName: \"kubernetes.io/projected/48a96c8d-490c-4283-8fcf-9c8e27cb3345-kube-api-access-5fgcw\") pod \"maas-controller-64fd4654b6-bsnch\" (UID: \"48a96c8d-490c-4283-8fcf-9c8e27cb3345\") " pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:54:19.249694 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:19.249668    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:54:19.368291 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:19.368250    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d5e01e9-aa7a-439c-a106-d616e12201a2-webhook-cert\") pod \"maas-controller-c6f6dcc69-w8vw4\" (UID: \"5d5e01e9-aa7a-439c-a106-d616e12201a2\") " pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:19.370582 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:19.370559    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d5e01e9-aa7a-439c-a106-d616e12201a2-webhook-cert\") pod \"maas-controller-c6f6dcc69-w8vw4\" (UID: \"5d5e01e9-aa7a-439c-a106-d616e12201a2\") " pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:19.381977 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:19.381957    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-bsnch"]
Jul 17 05:54:19.383836 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:54:19.383814    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a96c8d_490c_4283_8fcf_9c8e27cb3345.slice/crio-c1819b92593c27079a0a825e4b87fb8f5388f953f00d92264c19cd0b756f4817 WatchSource:0}: Error finding container c1819b92593c27079a0a825e4b87fb8f5388f953f00d92264c19cd0b756f4817: Status 404 returned error can't find the container with id c1819b92593c27079a0a825e4b87fb8f5388f953f00d92264c19cd0b756f4817
Jul 17 05:54:19.669671 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:19.669631    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:19.799571 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:19.799549    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c6f6dcc69-w8vw4"]
Jul 17 05:54:19.801514 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:54:19.801487    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d5e01e9_aa7a_439c_a106_d616e12201a2.slice/crio-69a2ca5261d72af0ef584a64d4d82c130011dbbe24a3a1d648eaea50fb5de11f WatchSource:0}: Error finding container 69a2ca5261d72af0ef584a64d4d82c130011dbbe24a3a1d648eaea50fb5de11f: Status 404 returned error can't find the container with id 69a2ca5261d72af0ef584a64d4d82c130011dbbe24a3a1d648eaea50fb5de11f
Jul 17 05:54:20.049522 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:20.049444    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4" event={"ID":"5d5e01e9-aa7a-439c-a106-d616e12201a2","Type":"ContainerStarted","Data":"69a2ca5261d72af0ef584a64d4d82c130011dbbe24a3a1d648eaea50fb5de11f"}
Jul 17 05:54:20.050518 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:20.050488    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fd4654b6-bsnch" event={"ID":"48a96c8d-490c-4283-8fcf-9c8e27cb3345","Type":"ContainerStarted","Data":"c1819b92593c27079a0a825e4b87fb8f5388f953f00d92264c19cd0b756f4817"}
Jul 17 05:54:23.070890 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:23.070854    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4" event={"ID":"5d5e01e9-aa7a-439c-a106-d616e12201a2","Type":"ContainerStarted","Data":"7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308"}
Jul 17 05:54:23.071334 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:23.070953    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:23.072498 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:23.072458    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fd4654b6-bsnch" event={"ID":"48a96c8d-490c-4283-8fcf-9c8e27cb3345","Type":"ContainerStarted","Data":"bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371"}
Jul 17 05:54:23.072617 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:23.072564    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:54:23.086999 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:23.086958    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4" podStartSLOduration=2.588726491 podStartE2EDuration="5.08694627s" totalImagesPullingTime="2.498219779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:54:23.086434468 +0000 UTC m=+785.528373043" watchObservedRunningTime="2026-07-17 05:54:23.08694627 +0000 UTC m=+785.528884849"
Jul 17 05:54:23.104418 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:23.104379    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64fd4654b6-bsnch" podStartSLOduration=2.197655429 podStartE2EDuration="5.104368399s" totalImagesPullingTime="2.90671297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:54:23.102594127 +0000 UTC m=+785.544532715" watchObservedRunningTime="2026-07-17 05:54:23.104368399 +0000 UTC m=+785.546306971"
Jul 17 05:54:34.079878 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:34.079849    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:34.080351 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:34.080051    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:54:34.135008 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:34.134979    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c6f6dcc69-w8vw4"]
Jul 17 05:54:34.135198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:34.135158    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4" podUID="5d5e01e9-aa7a-439c-a106-d616e12201a2" containerName="manager" containerID="cri-o://7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308" gracePeriod=30
Jul 17 05:54:34.379811 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:34.379788    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:34.484292 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:34.484270    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjbvh\" (UniqueName: \"kubernetes.io/projected/5d5e01e9-aa7a-439c-a106-d616e12201a2-kube-api-access-vjbvh\") pod \"5d5e01e9-aa7a-439c-a106-d616e12201a2\" (UID: \"5d5e01e9-aa7a-439c-a106-d616e12201a2\") "
Jul 17 05:54:34.484406 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:34.484342    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d5e01e9-aa7a-439c-a106-d616e12201a2-webhook-cert\") pod \"5d5e01e9-aa7a-439c-a106-d616e12201a2\" (UID: \"5d5e01e9-aa7a-439c-a106-d616e12201a2\") "
Jul 17 05:54:34.488540 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:34.487451    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5e01e9-aa7a-439c-a106-d616e12201a2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5d5e01e9-aa7a-439c-a106-d616e12201a2" (UID: "5d5e01e9-aa7a-439c-a106-d616e12201a2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:54:34.488540 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:34.487459    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d5e01e9-aa7a-439c-a106-d616e12201a2-kube-api-access-vjbvh" (OuterVolumeSpecName: "kube-api-access-vjbvh") pod "5d5e01e9-aa7a-439c-a106-d616e12201a2" (UID: "5d5e01e9-aa7a-439c-a106-d616e12201a2"). InnerVolumeSpecName "kube-api-access-vjbvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:54:34.585740 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:34.585710    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d5e01e9-aa7a-439c-a106-d616e12201a2-webhook-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:54:34.585740 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:34.585738    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjbvh\" (UniqueName: \"kubernetes.io/projected/5d5e01e9-aa7a-439c-a106-d616e12201a2-kube-api-access-vjbvh\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:54:35.129657 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:35.129620    2558 generic.go:356] "Generic (PLEG): container finished" podID="5d5e01e9-aa7a-439c-a106-d616e12201a2" containerID="7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308" exitCode=0
Jul 17 05:54:35.130089 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:35.129694    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4"
Jul 17 05:54:35.130089 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:35.129707    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4" event={"ID":"5d5e01e9-aa7a-439c-a106-d616e12201a2","Type":"ContainerDied","Data":"7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308"}
Jul 17 05:54:35.130089 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:35.129762    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c6f6dcc69-w8vw4" event={"ID":"5d5e01e9-aa7a-439c-a106-d616e12201a2","Type":"ContainerDied","Data":"69a2ca5261d72af0ef584a64d4d82c130011dbbe24a3a1d648eaea50fb5de11f"}
Jul 17 05:54:35.130089 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:35.129785    2558 scope.go:117] "RemoveContainer" containerID="7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308"
Jul 17 05:54:35.138701 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:35.138558    2558 scope.go:117] "RemoveContainer" containerID="7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308"
Jul 17 05:54:35.138835 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:54:35.138818    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308\": container with ID starting with 7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308 not found: ID does not exist" containerID="7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308"
Jul 17 05:54:35.138876 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:35.138842    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308"} err="failed to get container status \"7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308\": rpc error: code = NotFound desc = could not find container \"7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308\": container with ID starting with 7db6b333d9902a0f2f560b01e52b7c397608f2f959f5db991ab0f26a0aefd308 not found: ID does not exist"
Jul 17 05:54:35.151273 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:35.151244    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c6f6dcc69-w8vw4"]
Jul 17 05:54:35.154937 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:35.154916    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c6f6dcc69-w8vw4"]
Jul 17 05:54:36.078305 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:36.078275    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d5e01e9-aa7a-439c-a106-d616e12201a2" path="/var/lib/kubelet/pods/5d5e01e9-aa7a-439c-a106-d616e12201a2/volumes"
Jul 17 05:54:55.678541 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.678508    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"]
Jul 17 05:54:55.679025 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.679008    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5e01e9-aa7a-439c-a106-d616e12201a2" containerName="manager"
Jul 17 05:54:55.679072 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.679027    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5e01e9-aa7a-439c-a106-d616e12201a2" containerName="manager"
Jul 17 05:54:55.679122 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.679113    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5e01e9-aa7a-439c-a106-d616e12201a2" containerName="manager"
Jul 17 05:54:55.684448 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.684428    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:55.686980 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.686959    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 17 05:54:55.688418 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.688391    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 05:54:55.688565 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.688445    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 05:54:55.688565 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.688453    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-llpsw\""
Jul 17 05:54:55.690837 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.690814    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"]
Jul 17 05:54:55.754040 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.754014    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv4vn\" (UniqueName: \"kubernetes.io/projected/a5f8b487-3631-4e74-995c-fff6d904423d-kube-api-access-bv4vn\") pod \"maas-api-5bf885b94d-r9x75\" (UID: \"a5f8b487-3631-4e74-995c-fff6d904423d\") " pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:55.754159 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.754069    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5f8b487-3631-4e74-995c-fff6d904423d-maas-api-tls\") pod \"maas-api-5bf885b94d-r9x75\" (UID: \"a5f8b487-3631-4e74-995c-fff6d904423d\") " pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:55.754159 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.754109    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f8b487-3631-4e74-995c-fff6d904423d-openshift-service-ca\") pod \"maas-api-5bf885b94d-r9x75\" (UID: \"a5f8b487-3631-4e74-995c-fff6d904423d\") " pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:55.855442 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.855414    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5f8b487-3631-4e74-995c-fff6d904423d-maas-api-tls\") pod \"maas-api-5bf885b94d-r9x75\" (UID: \"a5f8b487-3631-4e74-995c-fff6d904423d\") " pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:55.855442 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.855444    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f8b487-3631-4e74-995c-fff6d904423d-openshift-service-ca\") pod \"maas-api-5bf885b94d-r9x75\" (UID: \"a5f8b487-3631-4e74-995c-fff6d904423d\") " pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:55.855623 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.855513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv4vn\" (UniqueName: \"kubernetes.io/projected/a5f8b487-3631-4e74-995c-fff6d904423d-kube-api-access-bv4vn\") pod \"maas-api-5bf885b94d-r9x75\" (UID: \"a5f8b487-3631-4e74-995c-fff6d904423d\") " pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:55.856163 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.856147    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5f8b487-3631-4e74-995c-fff6d904423d-openshift-service-ca\") pod \"maas-api-5bf885b94d-r9x75\" (UID: \"a5f8b487-3631-4e74-995c-fff6d904423d\") " pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:55.857957 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.857941    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5f8b487-3631-4e74-995c-fff6d904423d-maas-api-tls\") pod \"maas-api-5bf885b94d-r9x75\" (UID: \"a5f8b487-3631-4e74-995c-fff6d904423d\") " pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:55.863750 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.863733    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv4vn\" (UniqueName: \"kubernetes.io/projected/a5f8b487-3631-4e74-995c-fff6d904423d-kube-api-access-bv4vn\") pod \"maas-api-5bf885b94d-r9x75\" (UID: \"a5f8b487-3631-4e74-995c-fff6d904423d\") " pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:55.996086 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:55.996020    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:56.164575 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:56.164553    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"]
Jul 17 05:54:56.166942 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:54:56.166914    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f8b487_3631_4e74_995c_fff6d904423d.slice/crio-1cc4f325c8cc8995e21ffd3b1fde3748c16c3a519ac95b82ba8e28346f6d93ac WatchSource:0}: Error finding container 1cc4f325c8cc8995e21ffd3b1fde3748c16c3a519ac95b82ba8e28346f6d93ac: Status 404 returned error can't find the container with id 1cc4f325c8cc8995e21ffd3b1fde3748c16c3a519ac95b82ba8e28346f6d93ac
Jul 17 05:54:56.228923 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:56.228884    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75" event={"ID":"a5f8b487-3631-4e74-995c-fff6d904423d","Type":"ContainerStarted","Data":"1cc4f325c8cc8995e21ffd3b1fde3748c16c3a519ac95b82ba8e28346f6d93ac"}
Jul 17 05:54:58.256148 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:58.256111    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75" event={"ID":"a5f8b487-3631-4e74-995c-fff6d904423d","Type":"ContainerStarted","Data":"1437dac9a68ce51064424c4eb97281e1d987e29e6b99f6477d0ae464de327246"}
Jul 17 05:54:58.256602 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:58.256327    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:54:58.272711 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:54:58.272665    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75" podStartSLOduration=1.74481851 podStartE2EDuration="3.272650812s" totalImagesPullingTime="1.527832302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:54:58.270963732 +0000 UTC m=+820.712902341" watchObservedRunningTime="2026-07-17 05:54:58.272650812 +0000 UTC m=+820.714589384"
Jul 17 05:55:04.266546 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:04.266518    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5bf885b94d-r9x75"
Jul 17 05:55:04.289196 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:04.289168    2558 generic.go:356] "Generic (PLEG): container finished" podID="8cc064b1-bea1-41de-9e44-b10de3e4c1b3" containerID="a9370e0d88f0b24ab2b1d82f34348146c2a9dd9ee225ca7b7aedfb92a539144e" exitCode=0
Jul 17 05:55:04.289346 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:04.289247    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5" event={"ID":"8cc064b1-bea1-41de-9e44-b10de3e4c1b3","Type":"ContainerDied","Data":"a9370e0d88f0b24ab2b1d82f34348146c2a9dd9ee225ca7b7aedfb92a539144e"}
Jul 17 05:55:04.289660 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:04.289643    2558 scope.go:117] "RemoveContainer" containerID="a9370e0d88f0b24ab2b1d82f34348146c2a9dd9ee225ca7b7aedfb92a539144e"
Jul 17 05:55:04.290914 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:04.290886    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 05:55:04.290993 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:04.290950    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 05:55:04.290993 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:04.290988    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 05:55:05.296424 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:05.296389    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5" event={"ID":"8cc064b1-bea1-41de-9e44-b10de3e4c1b3","Type":"ContainerStarted","Data":"4dad6f4cc1607e2a5dc45a2215d89cc33d9fa5ec8de878663597624ecd4afda7"}
Jul 17 05:55:05.423680 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:05.423640    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:55:05.423964 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:05.423833    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g2wg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 17 05:55:05.424134 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:05.424111    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5" podUID="8cc064b1-bea1-41de-9e44-b10de3e4c1b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 17 05:55:06.424306 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:06.424268    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g2wg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 17 05:55:06.424789 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:06.424344    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5" podUID="8cc064b1-bea1-41de-9e44-b10de3e4c1b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 17 05:55:07.424505 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:07.424446    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g2wg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 17 05:55:07.425073 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:07.425038    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5" podUID="8cc064b1-bea1-41de-9e44-b10de3e4c1b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 17 05:55:08.424557 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:08.424514    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g2wg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 17 05:55:08.425052 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:08.424579    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5" podUID="8cc064b1-bea1-41de-9e44-b10de3e4c1b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 17 05:55:09.426831 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:09.426802    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:55:09.427288 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:09.427082    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:55:09.427894 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:55:09.427876    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g2wg5"
Jul 17 05:56:05.286148 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.286112    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"]
Jul 17 05:56:05.290061 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.290041    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.292817 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.292797    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 05:56:05.293934 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.293917    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 05:56:05.294059 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.294041    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8bkch\""
Jul 17 05:56:05.294146 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.294131    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 17 05:56:05.298417 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.298397    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"]
Jul 17 05:56:05.444383 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.444351    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.444558 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.444406    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.444558 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.444462    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.444558 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.444524    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.444731 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.444562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr2vd\" (UniqueName: \"kubernetes.io/projected/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-kube-api-access-kr2vd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.444731 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.444677    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.545393 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.545319    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.545393 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.545365    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.545640 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.545402    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.545640 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.545424    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.545640 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.545440    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.545640 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.545506    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr2vd\" (UniqueName: \"kubernetes.io/projected/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-kube-api-access-kr2vd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.545877 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.545850    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.546028 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.545913    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.546028 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.545942    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.547836 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.547812    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.548215 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.548197    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.553394 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.553366    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr2vd\" (UniqueName: \"kubernetes.io/projected/ab56ad30-c4d7-4b80-91ad-48d7bc5c9666-kube-api-access-kr2vd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qskbz\" (UID: \"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.620435 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.620408    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:05.757972 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:05.757948    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"]
Jul 17 05:56:05.759066 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:56:05.759046    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab56ad30_c4d7_4b80_91ad_48d7bc5c9666.slice/crio-22f46bdd1521254f59b051c7460e81c35c1d978e7b9debd1f48bdd1c2e3bf786 WatchSource:0}: Error finding container 22f46bdd1521254f59b051c7460e81c35c1d978e7b9debd1f48bdd1c2e3bf786: Status 404 returned error can't find the container with id 22f46bdd1521254f59b051c7460e81c35c1d978e7b9debd1f48bdd1c2e3bf786
Jul 17 05:56:06.594708 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:06.594659    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz" event={"ID":"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666","Type":"ContainerStarted","Data":"22f46bdd1521254f59b051c7460e81c35c1d978e7b9debd1f48bdd1c2e3bf786"}
Jul 17 05:56:07.601350 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:07.601315    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz" event={"ID":"ab56ad30-c4d7-4b80-91ad-48d7bc5c9666","Type":"ContainerStarted","Data":"b3ca96f9a87c8c397aadcca93713873e89477a775da9d623f96896e6e6de1d4f"}
Jul 17 05:56:07.620285 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:07.620240    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz" podStartSLOduration=1.608038365 podStartE2EDuration="2.620222121s" totalImagesPullingTime="1.012183756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:56:07.61795322 +0000 UTC m=+890.059891793" watchObservedRunningTime="2026-07-17 05:56:07.620222121 +0000 UTC m=+890.062160696"
Jul 17 05:56:15.621528 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:15.621415    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:15.621528 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:15.621501    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:15.637273 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:15.637243    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:15.655554 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:15.655530    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qskbz"
Jul 17 05:56:17.763856 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.763818    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"]
Jul 17 05:56:17.766846 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.766825    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.769529 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.769504    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 17 05:56:17.777399 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.777376    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"]
Jul 17 05:56:17.851837 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.851798    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da6c99ee-9017-437f-a0eb-cac708960f9a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.851988 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.851840    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.851988 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.851878    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.851988 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.851910    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlgw2\" (UniqueName: \"kubernetes.io/projected/da6c99ee-9017-437f-a0eb-cac708960f9a-kube-api-access-qlgw2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.851988 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.851963    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.852381 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.852050    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.953331 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.953298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.953331 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.953331    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da6c99ee-9017-437f-a0eb-cac708960f9a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.953616 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.953354    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.953616 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.953375    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.953616 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.953403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlgw2\" (UniqueName: \"kubernetes.io/projected/da6c99ee-9017-437f-a0eb-cac708960f9a-kube-api-access-qlgw2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.953616 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.953427    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.953833 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.953707    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.953833 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.953739    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.953833 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.953759    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.955709 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.955690    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da6c99ee-9017-437f-a0eb-cac708960f9a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.956088 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.956074    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 17 05:56:17.961721 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.961702    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlgw2\" (UniqueName: \"kubernetes.io/projected/da6c99ee-9017-437f-a0eb-cac708960f9a-kube-api-access-qlgw2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.966606 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.966582    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da6c99ee-9017-437f-a0eb-cac708960f9a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh\" (UID: \"da6c99ee-9017-437f-a0eb-cac708960f9a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:17.983135 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.983116    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 05:56:17.984362 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.984342    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 05:56:17.988504 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.988487    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 05:56:17.990055 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:17.990038    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 05:56:18.078973 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:18.078896    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:18.217818 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:18.217796    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"]
Jul 17 05:56:18.219334 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:56:18.219289    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda6c99ee_9017_437f_a0eb_cac708960f9a.slice/crio-0a2f6f45423d20742322205cb840ba842741f5c7a2b8f578aefe60657138de2c WatchSource:0}: Error finding container 0a2f6f45423d20742322205cb840ba842741f5c7a2b8f578aefe60657138de2c: Status 404 returned error can't find the container with id 0a2f6f45423d20742322205cb840ba842741f5c7a2b8f578aefe60657138de2c
Jul 17 05:56:18.661981 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:18.661898    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh" event={"ID":"da6c99ee-9017-437f-a0eb-cac708960f9a","Type":"ContainerStarted","Data":"f61110a29f9bbb4ae3f48703e9bad3880056b71636ccef8d5fcca3b7e19be692"}
Jul 17 05:56:18.661981 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:18.661950    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh" event={"ID":"da6c99ee-9017-437f-a0eb-cac708960f9a","Type":"ContainerStarted","Data":"0a2f6f45423d20742322205cb840ba842741f5c7a2b8f578aefe60657138de2c"}
Jul 17 05:56:18.680208 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:18.680159    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh" podStartSLOduration=1.505993595 podStartE2EDuration="1.68014376s" totalImagesPullingTime="174.150165ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:56:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:56:18.677467375 +0000 UTC m=+901.119405949" watchObservedRunningTime="2026-07-17 05:56:18.68014376 +0000 UTC m=+901.122082360"
Jul 17 05:56:28.079427 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:28.079395    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:28.079821 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:28.079466    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:28.091533 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:28.091512    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:56:28.725919 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:56:28.725895    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh"
Jul 17 05:58:10.841757 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:10.841727    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-749f4c9f87-4mdml"]
Jul 17 05:58:10.842295 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:10.841951    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-749f4c9f87-4mdml" podUID="30ed9ccc-997f-411d-9e85-7c8d2230d6a1" containerName="authorino" containerID="cri-o://7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78" gracePeriod=30
Jul 17 05:58:11.089543 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.089520    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-749f4c9f87-4mdml"
Jul 17 05:58:11.169299 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.169237    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmgvg\" (UniqueName: \"kubernetes.io/projected/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-kube-api-access-qmgvg\") pod \"30ed9ccc-997f-411d-9e85-7c8d2230d6a1\" (UID: \"30ed9ccc-997f-411d-9e85-7c8d2230d6a1\") "
Jul 17 05:58:11.169415 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.169304    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-tls-cert\") pod \"30ed9ccc-997f-411d-9e85-7c8d2230d6a1\" (UID: \"30ed9ccc-997f-411d-9e85-7c8d2230d6a1\") "
Jul 17 05:58:11.171916 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.171891    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-kube-api-access-qmgvg" (OuterVolumeSpecName: "kube-api-access-qmgvg") pod "30ed9ccc-997f-411d-9e85-7c8d2230d6a1" (UID: "30ed9ccc-997f-411d-9e85-7c8d2230d6a1"). InnerVolumeSpecName "kube-api-access-qmgvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:58:11.180903 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.180880    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "30ed9ccc-997f-411d-9e85-7c8d2230d6a1" (UID: "30ed9ccc-997f-411d-9e85-7c8d2230d6a1"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:58:11.224733 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.224703    2558 generic.go:356] "Generic (PLEG): container finished" podID="30ed9ccc-997f-411d-9e85-7c8d2230d6a1" containerID="7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78" exitCode=0
Jul 17 05:58:11.224854 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.224737    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-749f4c9f87-4mdml"
Jul 17 05:58:11.224854 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.224798    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-749f4c9f87-4mdml" event={"ID":"30ed9ccc-997f-411d-9e85-7c8d2230d6a1","Type":"ContainerDied","Data":"7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78"}
Jul 17 05:58:11.224854 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.224840    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-749f4c9f87-4mdml" event={"ID":"30ed9ccc-997f-411d-9e85-7c8d2230d6a1","Type":"ContainerDied","Data":"980a3215396cb41c7d2c12f5cdc6962a7b455755454281cbe720cfb630fe3dc2"}
Jul 17 05:58:11.225028 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.224860    2558 scope.go:117] "RemoveContainer" containerID="7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78"
Jul 17 05:58:11.235149 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.235123    2558 scope.go:117] "RemoveContainer" containerID="7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78"
Jul 17 05:58:11.235394 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:58:11.235377    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78\": container with ID starting with 7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78 not found: ID does not exist" containerID="7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78"
Jul 17 05:58:11.235447 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.235401    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78"} err="failed to get container status \"7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78\": rpc error: code = NotFound desc = could not find container \"7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78\": container with ID starting with 7fcc91546d7cc594475a5b33928696221e3ab5f38b3c6c1124d26d887d0e3e78 not found: ID does not exist"
Jul 17 05:58:11.246560 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.246532    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-749f4c9f87-4mdml"]
Jul 17 05:58:11.250012 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.249992    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-749f4c9f87-4mdml"]
Jul 17 05:58:11.270521 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.270500    2558 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-tls-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:58:11.270584 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:11.270524    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmgvg\" (UniqueName: \"kubernetes.io/projected/30ed9ccc-997f-411d-9e85-7c8d2230d6a1-kube-api-access-qmgvg\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:58:12.078710 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:12.078644    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30ed9ccc-997f-411d-9e85-7c8d2230d6a1" path="/var/lib/kubelet/pods/30ed9ccc-997f-411d-9e85-7c8d2230d6a1/volumes"
Jul 17 05:58:55.656403 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:55.656368    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-bsnch"]
Jul 17 05:58:55.656895 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:55.656647    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64fd4654b6-bsnch" podUID="48a96c8d-490c-4283-8fcf-9c8e27cb3345" containerName="manager" containerID="cri-o://bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371" gracePeriod=30
Jul 17 05:58:55.903411 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:55.903389    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:58:56.030306 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.030241    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48a96c8d-490c-4283-8fcf-9c8e27cb3345-webhook-cert\") pod \"48a96c8d-490c-4283-8fcf-9c8e27cb3345\" (UID: \"48a96c8d-490c-4283-8fcf-9c8e27cb3345\") "
Jul 17 05:58:56.030306 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.030301    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fgcw\" (UniqueName: \"kubernetes.io/projected/48a96c8d-490c-4283-8fcf-9c8e27cb3345-kube-api-access-5fgcw\") pod \"48a96c8d-490c-4283-8fcf-9c8e27cb3345\" (UID: \"48a96c8d-490c-4283-8fcf-9c8e27cb3345\") "
Jul 17 05:58:56.032484 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.032434    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48a96c8d-490c-4283-8fcf-9c8e27cb3345-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "48a96c8d-490c-4283-8fcf-9c8e27cb3345" (UID: "48a96c8d-490c-4283-8fcf-9c8e27cb3345"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:58:56.032585 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.032510    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48a96c8d-490c-4283-8fcf-9c8e27cb3345-kube-api-access-5fgcw" (OuterVolumeSpecName: "kube-api-access-5fgcw") pod "48a96c8d-490c-4283-8fcf-9c8e27cb3345" (UID: "48a96c8d-490c-4283-8fcf-9c8e27cb3345"). InnerVolumeSpecName "kube-api-access-5fgcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:58:56.130863 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.130838    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48a96c8d-490c-4283-8fcf-9c8e27cb3345-webhook-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:58:56.130863 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.130860    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fgcw\" (UniqueName: \"kubernetes.io/projected/48a96c8d-490c-4283-8fcf-9c8e27cb3345-kube-api-access-5fgcw\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 05:58:56.455021 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.454982    2558 generic.go:356] "Generic (PLEG): container finished" podID="48a96c8d-490c-4283-8fcf-9c8e27cb3345" containerID="bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371" exitCode=0
Jul 17 05:58:56.455178 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.455030    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fd4654b6-bsnch"
Jul 17 05:58:56.455178 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.455063    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fd4654b6-bsnch" event={"ID":"48a96c8d-490c-4283-8fcf-9c8e27cb3345","Type":"ContainerDied","Data":"bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371"}
Jul 17 05:58:56.455178 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.455125    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fd4654b6-bsnch" event={"ID":"48a96c8d-490c-4283-8fcf-9c8e27cb3345","Type":"ContainerDied","Data":"c1819b92593c27079a0a825e4b87fb8f5388f953f00d92264c19cd0b756f4817"}
Jul 17 05:58:56.455178 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.455148    2558 scope.go:117] "RemoveContainer" containerID="bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371"
Jul 17 05:58:56.463962 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.463945    2558 scope.go:117] "RemoveContainer" containerID="bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371"
Jul 17 05:58:56.464200 ip-10-0-140-96 kubenswrapper[2558]: E0717 05:58:56.464184    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371\": container with ID starting with bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371 not found: ID does not exist" containerID="bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371"
Jul 17 05:58:56.464248 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.464209    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371"} err="failed to get container status \"bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371\": rpc error: code = NotFound desc = could not find container \"bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371\": container with ID starting with bffdea7347d0861717a983c3597bafaef885b6aac8955d512ef0e14c9453c371 not found: ID does not exist"
Jul 17 05:58:56.471065 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.471043    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-bsnch"]
Jul 17 05:58:56.474569 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:56.474551    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-bsnch"]
Jul 17 05:58:58.078304 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:58:58.078273    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48a96c8d-490c-4283-8fcf-9c8e27cb3345" path="/var/lib/kubelet/pods/48a96c8d-490c-4283-8fcf-9c8e27cb3345/volumes"
Jul 17 05:59:57.262593 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.262557    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64fd4654b6-q6rlw"]
Jul 17 05:59:57.263198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.262958    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48a96c8d-490c-4283-8fcf-9c8e27cb3345" containerName="manager"
Jul 17 05:59:57.263198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.262969    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="48a96c8d-490c-4283-8fcf-9c8e27cb3345" containerName="manager"
Jul 17 05:59:57.263198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.262985    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30ed9ccc-997f-411d-9e85-7c8d2230d6a1" containerName="authorino"
Jul 17 05:59:57.263198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.262991    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="30ed9ccc-997f-411d-9e85-7c8d2230d6a1" containerName="authorino"
Jul 17 05:59:57.263198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.263049    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="30ed9ccc-997f-411d-9e85-7c8d2230d6a1" containerName="authorino"
Jul 17 05:59:57.263198 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.263058    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="48a96c8d-490c-4283-8fcf-9c8e27cb3345" containerName="manager"
Jul 17 05:59:57.266149 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.266129    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 05:59:57.268661 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.268633    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 05:59:57.269677 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.269660    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 05:59:57.269766 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.269709    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9ncjx\""
Jul 17 05:59:57.275955 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.275926    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-q6rlw"]
Jul 17 05:59:57.306181 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.306157    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qkrb\" (UniqueName: \"kubernetes.io/projected/9867a92e-144c-46a1-b6bb-94313cd5302a-kube-api-access-4qkrb\") pod \"maas-controller-64fd4654b6-q6rlw\" (UID: \"9867a92e-144c-46a1-b6bb-94313cd5302a\") " pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 05:59:57.306301 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.306231    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9867a92e-144c-46a1-b6bb-94313cd5302a-webhook-cert\") pod \"maas-controller-64fd4654b6-q6rlw\" (UID: \"9867a92e-144c-46a1-b6bb-94313cd5302a\") " pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 05:59:57.407603 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.407577    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9867a92e-144c-46a1-b6bb-94313cd5302a-webhook-cert\") pod \"maas-controller-64fd4654b6-q6rlw\" (UID: \"9867a92e-144c-46a1-b6bb-94313cd5302a\") " pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 05:59:57.407702 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.407631    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qkrb\" (UniqueName: \"kubernetes.io/projected/9867a92e-144c-46a1-b6bb-94313cd5302a-kube-api-access-4qkrb\") pod \"maas-controller-64fd4654b6-q6rlw\" (UID: \"9867a92e-144c-46a1-b6bb-94313cd5302a\") " pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 05:59:57.409867 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.409845    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9867a92e-144c-46a1-b6bb-94313cd5302a-webhook-cert\") pod \"maas-controller-64fd4654b6-q6rlw\" (UID: \"9867a92e-144c-46a1-b6bb-94313cd5302a\") " pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 05:59:57.415566 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.415545    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qkrb\" (UniqueName: \"kubernetes.io/projected/9867a92e-144c-46a1-b6bb-94313cd5302a-kube-api-access-4qkrb\") pod \"maas-controller-64fd4654b6-q6rlw\" (UID: \"9867a92e-144c-46a1-b6bb-94313cd5302a\") " pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 05:59:57.578231 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.578182    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 05:59:57.708428 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.708407    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-q6rlw"]
Jul 17 05:59:57.710663 ip-10-0-140-96 kubenswrapper[2558]: W0717 05:59:57.710633    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9867a92e_144c_46a1_b6bb_94313cd5302a.slice/crio-3b4ad19d2100d31881438c3b86edda35da2f84a8ed418c8b469baef9c77e1583 WatchSource:0}: Error finding container 3b4ad19d2100d31881438c3b86edda35da2f84a8ed418c8b469baef9c77e1583: Status 404 returned error can't find the container with id 3b4ad19d2100d31881438c3b86edda35da2f84a8ed418c8b469baef9c77e1583
Jul 17 05:59:57.712023 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.712005    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:59:57.755112 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:57.755085    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fd4654b6-q6rlw" event={"ID":"9867a92e-144c-46a1-b6bb-94313cd5302a","Type":"ContainerStarted","Data":"3b4ad19d2100d31881438c3b86edda35da2f84a8ed418c8b469baef9c77e1583"}
Jul 17 05:59:58.761193 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:58.761159    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fd4654b6-q6rlw" event={"ID":"9867a92e-144c-46a1-b6bb-94313cd5302a","Type":"ContainerStarted","Data":"a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0"}
Jul 17 05:59:58.761595 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:58.761230    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 05:59:58.778071 ip-10-0-140-96 kubenswrapper[2558]: I0717 05:59:58.778023    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64fd4654b6-q6rlw" podStartSLOduration=1.28073758 podStartE2EDuration="1.778008276s" totalImagesPullingTime="497.270696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:59:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:59:58.776227835 +0000 UTC m=+1121.218166411" watchObservedRunningTime="2026-07-17 05:59:58.778008276 +0000 UTC m=+1121.219946850"
Jul 17 06:00:00.139916 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:00.139892    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737800-vqlhm"]
Jul 17 06:00:00.143164 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:00.143150    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737800-vqlhm"
Jul 17 06:00:00.145855 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:00.145834    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:00:00.147809 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:00.147790    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737800-vqlhm"]
Jul 17 06:00:00.155094 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:00.155078    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737800-vqlhm"
Jul 17 06:00:00.290832 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:00:00.290788    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a303bfb_b28a_4e61_a5cf_cf6e2742f10d.slice/crio-17d9d1b51f2e8de24dad8e454aebd5e471b9325d50431b01b79ebf003ab5421c WatchSource:0}: Error finding container 17d9d1b51f2e8de24dad8e454aebd5e471b9325d50431b01b79ebf003ab5421c: Status 404 returned error can't find the container with id 17d9d1b51f2e8de24dad8e454aebd5e471b9325d50431b01b79ebf003ab5421c
Jul 17 06:00:00.293274 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:00.293251    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737800-vqlhm"]
Jul 17 06:00:00.772861 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:00.772827    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737800-vqlhm" event={"ID":"4a303bfb-b28a-4e61-a5cf-cf6e2742f10d","Type":"ContainerStarted","Data":"17d9d1b51f2e8de24dad8e454aebd5e471b9325d50431b01b79ebf003ab5421c"}
Jul 17 06:00:05.799813 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:05.799726    2558 generic.go:356] "Generic (PLEG): container finished" podID="4a303bfb-b28a-4e61-a5cf-cf6e2742f10d" containerID="4452c1c8d7f3b055ffbec7d211ac216903a91313f11a2ede345d5286cf95d54e" exitCode=0
Jul 17 06:00:05.800194 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:05.799809    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737800-vqlhm" event={"ID":"4a303bfb-b28a-4e61-a5cf-cf6e2742f10d","Type":"ContainerDied","Data":"4452c1c8d7f3b055ffbec7d211ac216903a91313f11a2ede345d5286cf95d54e"}
Jul 17 06:00:06.940372 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:06.940350    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737800-vqlhm"
Jul 17 06:00:07.810610 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:07.810582    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737800-vqlhm"
Jul 17 06:00:07.810610 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:07.810586    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737800-vqlhm" event={"ID":"4a303bfb-b28a-4e61-a5cf-cf6e2742f10d","Type":"ContainerDied","Data":"17d9d1b51f2e8de24dad8e454aebd5e471b9325d50431b01b79ebf003ab5421c"}
Jul 17 06:00:07.810823 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:07.810633    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17d9d1b51f2e8de24dad8e454aebd5e471b9325d50431b01b79ebf003ab5421c"
Jul 17 06:00:09.768808 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:09.768778    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 06:00:14.392589 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.392553    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-q6rlw"]
Jul 17 06:00:14.393099 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.392786    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64fd4654b6-q6rlw" podUID="9867a92e-144c-46a1-b6bb-94313cd5302a" containerName="manager" containerID="cri-o://a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0" gracePeriod=30
Jul 17 06:00:14.646842 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.646789    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 06:00:14.756506 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.756458    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qkrb\" (UniqueName: \"kubernetes.io/projected/9867a92e-144c-46a1-b6bb-94313cd5302a-kube-api-access-4qkrb\") pod \"9867a92e-144c-46a1-b6bb-94313cd5302a\" (UID: \"9867a92e-144c-46a1-b6bb-94313cd5302a\") "
Jul 17 06:00:14.756674 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.756630    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9867a92e-144c-46a1-b6bb-94313cd5302a-webhook-cert\") pod \"9867a92e-144c-46a1-b6bb-94313cd5302a\" (UID: \"9867a92e-144c-46a1-b6bb-94313cd5302a\") "
Jul 17 06:00:14.758636 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.758607    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9867a92e-144c-46a1-b6bb-94313cd5302a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9867a92e-144c-46a1-b6bb-94313cd5302a" (UID: "9867a92e-144c-46a1-b6bb-94313cd5302a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 06:00:14.758636 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.758607    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9867a92e-144c-46a1-b6bb-94313cd5302a-kube-api-access-4qkrb" (OuterVolumeSpecName: "kube-api-access-4qkrb") pod "9867a92e-144c-46a1-b6bb-94313cd5302a" (UID: "9867a92e-144c-46a1-b6bb-94313cd5302a"). InnerVolumeSpecName "kube-api-access-4qkrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:00:14.849103 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.849068    2558 generic.go:356] "Generic (PLEG): container finished" podID="9867a92e-144c-46a1-b6bb-94313cd5302a" containerID="a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0" exitCode=0
Jul 17 06:00:14.849237 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.849115    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fd4654b6-q6rlw"
Jul 17 06:00:14.849237 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.849145    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fd4654b6-q6rlw" event={"ID":"9867a92e-144c-46a1-b6bb-94313cd5302a","Type":"ContainerDied","Data":"a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0"}
Jul 17 06:00:14.849237 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.849197    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fd4654b6-q6rlw" event={"ID":"9867a92e-144c-46a1-b6bb-94313cd5302a","Type":"ContainerDied","Data":"3b4ad19d2100d31881438c3b86edda35da2f84a8ed418c8b469baef9c77e1583"}
Jul 17 06:00:14.849237 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.849218    2558 scope.go:117] "RemoveContainer" containerID="a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0"
Jul 17 06:00:14.857310 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.857292    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9867a92e-144c-46a1-b6bb-94313cd5302a-webhook-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:00:14.857393 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.857312    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qkrb\" (UniqueName: \"kubernetes.io/projected/9867a92e-144c-46a1-b6bb-94313cd5302a-kube-api-access-4qkrb\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:00:14.858358 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.858345    2558 scope.go:117] "RemoveContainer" containerID="a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0"
Jul 17 06:00:14.858687 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:00:14.858667    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0\": container with ID starting with a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0 not found: ID does not exist" containerID="a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0"
Jul 17 06:00:14.858770 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.858699    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0"} err="failed to get container status \"a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0\": rpc error: code = NotFound desc = could not find container \"a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0\": container with ID starting with a88ce08fee574a03199ac7b8b369d7ee818eea8c45eefa22ec32f8eca62eb4e0 not found: ID does not exist"
Jul 17 06:00:14.868536 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.868515    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-q6rlw"]
Jul 17 06:00:14.873658 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:14.873640    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-q6rlw"]
Jul 17 06:00:15.674563 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.674529    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64fd4654b6-gcwf9"]
Jul 17 06:00:15.675053 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.675039    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9867a92e-144c-46a1-b6bb-94313cd5302a" containerName="manager"
Jul 17 06:00:15.675123 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.675059    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9867a92e-144c-46a1-b6bb-94313cd5302a" containerName="manager"
Jul 17 06:00:15.675123 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.675073    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a303bfb-b28a-4e61-a5cf-cf6e2742f10d" containerName="cleanup"
Jul 17 06:00:15.675123 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.675084    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a303bfb-b28a-4e61-a5cf-cf6e2742f10d" containerName="cleanup"
Jul 17 06:00:15.675231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.675156    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a303bfb-b28a-4e61-a5cf-cf6e2742f10d" containerName="cleanup"
Jul 17 06:00:15.675231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.675166    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9867a92e-144c-46a1-b6bb-94313cd5302a" containerName="manager"
Jul 17 06:00:15.678234 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.678218    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fd4654b6-gcwf9"
Jul 17 06:00:15.680574 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.680557    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 06:00:15.681709 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.681692    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 06:00:15.681785 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.681741    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9ncjx\""
Jul 17 06:00:15.687303 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.687283    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-gcwf9"]
Jul 17 06:00:15.765277 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.765249    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btswc\" (UniqueName: \"kubernetes.io/projected/54992e61-e79f-4679-92cd-f3028c00ff44-kube-api-access-btswc\") pod \"maas-controller-64fd4654b6-gcwf9\" (UID: \"54992e61-e79f-4679-92cd-f3028c00ff44\") " pod="opendatahub/maas-controller-64fd4654b6-gcwf9"
Jul 17 06:00:15.765412 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.765299    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54992e61-e79f-4679-92cd-f3028c00ff44-webhook-cert\") pod \"maas-controller-64fd4654b6-gcwf9\" (UID: \"54992e61-e79f-4679-92cd-f3028c00ff44\") " pod="opendatahub/maas-controller-64fd4654b6-gcwf9"
Jul 17 06:00:15.866275 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.866248    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btswc\" (UniqueName: \"kubernetes.io/projected/54992e61-e79f-4679-92cd-f3028c00ff44-kube-api-access-btswc\") pod \"maas-controller-64fd4654b6-gcwf9\" (UID: \"54992e61-e79f-4679-92cd-f3028c00ff44\") " pod="opendatahub/maas-controller-64fd4654b6-gcwf9"
Jul 17 06:00:15.866391 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.866289    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54992e61-e79f-4679-92cd-f3028c00ff44-webhook-cert\") pod \"maas-controller-64fd4654b6-gcwf9\" (UID: \"54992e61-e79f-4679-92cd-f3028c00ff44\") " pod="opendatahub/maas-controller-64fd4654b6-gcwf9"
Jul 17 06:00:15.868908 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.868890    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54992e61-e79f-4679-92cd-f3028c00ff44-webhook-cert\") pod \"maas-controller-64fd4654b6-gcwf9\" (UID: \"54992e61-e79f-4679-92cd-f3028c00ff44\") " pod="opendatahub/maas-controller-64fd4654b6-gcwf9"
Jul 17 06:00:15.874863 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.874842    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btswc\" (UniqueName: \"kubernetes.io/projected/54992e61-e79f-4679-92cd-f3028c00ff44-kube-api-access-btswc\") pod \"maas-controller-64fd4654b6-gcwf9\" (UID: \"54992e61-e79f-4679-92cd-f3028c00ff44\") " pod="opendatahub/maas-controller-64fd4654b6-gcwf9"
Jul 17 06:00:15.989598 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:15.989528    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64fd4654b6-gcwf9"
Jul 17 06:00:16.080498 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:16.080448    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9867a92e-144c-46a1-b6bb-94313cd5302a" path="/var/lib/kubelet/pods/9867a92e-144c-46a1-b6bb-94313cd5302a/volumes"
Jul 17 06:00:16.127827 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:16.127803    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64fd4654b6-gcwf9"]
Jul 17 06:00:16.129990 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:00:16.129962    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54992e61_e79f_4679_92cd_f3028c00ff44.slice/crio-f7cd0ad30aef9e1bea2a179260e4efdd1999ebfcf08b39fbcb2b49076a37bb31 WatchSource:0}: Error finding container f7cd0ad30aef9e1bea2a179260e4efdd1999ebfcf08b39fbcb2b49076a37bb31: Status 404 returned error can't find the container with id f7cd0ad30aef9e1bea2a179260e4efdd1999ebfcf08b39fbcb2b49076a37bb31
Jul 17 06:00:16.861620 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:16.861588    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fd4654b6-gcwf9" event={"ID":"54992e61-e79f-4679-92cd-f3028c00ff44","Type":"ContainerStarted","Data":"12ccd11b2cee61365f0f4f5f8210e5b69853bc58b4076e6fba55a0319ac69ba8"}
Jul 17 06:00:16.862065 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:16.861637    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64fd4654b6-gcwf9" event={"ID":"54992e61-e79f-4679-92cd-f3028c00ff44","Type":"ContainerStarted","Data":"f7cd0ad30aef9e1bea2a179260e4efdd1999ebfcf08b39fbcb2b49076a37bb31"}
Jul 17 06:00:16.862065 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:16.861717    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64fd4654b6-gcwf9"
Jul 17 06:00:16.877786 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:16.877610    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64fd4654b6-gcwf9" podStartSLOduration=1.558008696 podStartE2EDuration="1.877591663s" totalImagesPullingTime="319.582967ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:00:16.876821357 +0000 UTC m=+1139.318759932" watchObservedRunningTime="2026-07-17 06:00:16.877591663 +0000 UTC m=+1139.319530236"
Jul 17 06:00:27.869847 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:00:27.869817    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64fd4654b6-gcwf9"
Jul 17 06:01:18.013879 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:01:18.013846    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:01:18.016934 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:01:18.016910    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:01:18.018766 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:01:18.018750    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:01:18.021873 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:01:18.021858    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:05:04.509498 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:04.509353    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"]
Jul 17 06:05:04.512596 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:04.509666    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf" podUID="3214c6c9-d799-40c5-9acc-c3e294139f62" containerName="manager" containerID="cri-o://91588442de1696dafc473b9c3e41a997743887a8634af1199974bd499488abf9" gracePeriod=10
Jul 17 06:05:08.350073 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:08.350034    2558 generic.go:356] "Generic (PLEG): container finished" podID="3214c6c9-d799-40c5-9acc-c3e294139f62" containerID="91588442de1696dafc473b9c3e41a997743887a8634af1199974bd499488abf9" exitCode=0
Jul 17 06:05:08.350073 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:08.350073    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf" event={"ID":"3214c6c9-d799-40c5-9acc-c3e294139f62","Type":"ContainerDied","Data":"91588442de1696dafc473b9c3e41a997743887a8634af1199974bd499488abf9"}
Jul 17 06:05:08.773694 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:08.773659    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 06:05:08.890424 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:08.890318    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4tvq\" (UniqueName: \"kubernetes.io/projected/3214c6c9-d799-40c5-9acc-c3e294139f62-kube-api-access-m4tvq\") pod \"3214c6c9-d799-40c5-9acc-c3e294139f62\" (UID: \"3214c6c9-d799-40c5-9acc-c3e294139f62\") "
Jul 17 06:05:08.890645 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:08.890450    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3214c6c9-d799-40c5-9acc-c3e294139f62-extensions-socket-volume\") pod \"3214c6c9-d799-40c5-9acc-c3e294139f62\" (UID: \"3214c6c9-d799-40c5-9acc-c3e294139f62\") "
Jul 17 06:05:08.890869 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:08.890843    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3214c6c9-d799-40c5-9acc-c3e294139f62-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3214c6c9-d799-40c5-9acc-c3e294139f62" (UID: "3214c6c9-d799-40c5-9acc-c3e294139f62"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:05:08.892825 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:08.892796    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3214c6c9-d799-40c5-9acc-c3e294139f62-kube-api-access-m4tvq" (OuterVolumeSpecName: "kube-api-access-m4tvq") pod "3214c6c9-d799-40c5-9acc-c3e294139f62" (UID: "3214c6c9-d799-40c5-9acc-c3e294139f62"). InnerVolumeSpecName "kube-api-access-m4tvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:05:08.991367 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:08.991323    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3214c6c9-d799-40c5-9acc-c3e294139f62-extensions-socket-volume\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:05:08.991367 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:08.991357    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4tvq\" (UniqueName: \"kubernetes.io/projected/3214c6c9-d799-40c5-9acc-c3e294139f62-kube-api-access-m4tvq\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:05:09.356717 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:09.356675    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf" event={"ID":"3214c6c9-d799-40c5-9acc-c3e294139f62","Type":"ContainerDied","Data":"db8c14cc34df0914ab2d7288ddd29372410b6aa190e1e866a5488810d62df351"}
Jul 17 06:05:09.356717 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:09.356700    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"
Jul 17 06:05:09.357235 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:09.356751    2558 scope.go:117] "RemoveContainer" containerID="91588442de1696dafc473b9c3e41a997743887a8634af1199974bd499488abf9"
Jul 17 06:05:09.379431 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:09.379396    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"]
Jul 17 06:05:09.382986 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:09.382952    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p2wjf"]
Jul 17 06:05:10.079513 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:05:10.079461    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3214c6c9-d799-40c5-9acc-c3e294139f62" path="/var/lib/kubelet/pods/3214c6c9-d799-40c5-9acc-c3e294139f62/volumes"
Jul 17 06:06:05.670317 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.670278    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"]
Jul 17 06:06:05.670912 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.670881    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3214c6c9-d799-40c5-9acc-c3e294139f62" containerName="manager"
Jul 17 06:06:05.670912 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.670900    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3214c6c9-d799-40c5-9acc-c3e294139f62" containerName="manager"
Jul 17 06:06:05.671051 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.671029    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3214c6c9-d799-40c5-9acc-c3e294139f62" containerName="manager"
Jul 17 06:06:05.673954 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.673933    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"
Jul 17 06:06:05.676337 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.676320    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vgf6m\""
Jul 17 06:06:05.686398 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.686375    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"]
Jul 17 06:06:05.771294 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.771263    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz82r\" (UniqueName: \"kubernetes.io/projected/c03a52bc-8bfe-4543-8bef-0d29efde9b16-kube-api-access-gz82r\") pod \"kuadrant-operator-controller-manager-86878c769f-j6tkg\" (UID: \"c03a52bc-8bfe-4543-8bef-0d29efde9b16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"
Jul 17 06:06:05.771404 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.771331    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c03a52bc-8bfe-4543-8bef-0d29efde9b16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j6tkg\" (UID: \"c03a52bc-8bfe-4543-8bef-0d29efde9b16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"
Jul 17 06:06:05.872725 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.872705    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz82r\" (UniqueName: \"kubernetes.io/projected/c03a52bc-8bfe-4543-8bef-0d29efde9b16-kube-api-access-gz82r\") pod \"kuadrant-operator-controller-manager-86878c769f-j6tkg\" (UID: \"c03a52bc-8bfe-4543-8bef-0d29efde9b16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"
Jul 17 06:06:05.872839 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.872751    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c03a52bc-8bfe-4543-8bef-0d29efde9b16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j6tkg\" (UID: \"c03a52bc-8bfe-4543-8bef-0d29efde9b16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"
Jul 17 06:06:05.873054 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.873036    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c03a52bc-8bfe-4543-8bef-0d29efde9b16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j6tkg\" (UID: \"c03a52bc-8bfe-4543-8bef-0d29efde9b16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"
Jul 17 06:06:05.880626 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.880602    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz82r\" (UniqueName: \"kubernetes.io/projected/c03a52bc-8bfe-4543-8bef-0d29efde9b16-kube-api-access-gz82r\") pod \"kuadrant-operator-controller-manager-86878c769f-j6tkg\" (UID: \"c03a52bc-8bfe-4543-8bef-0d29efde9b16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"
Jul 17 06:06:05.984563 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:05.984500    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"
Jul 17 06:06:06.127144 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:06.127123    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"]
Jul 17 06:06:06.128652 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:06:06.128624    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc03a52bc_8bfe_4543_8bef_0d29efde9b16.slice/crio-2556b9e916acfda1dcb9d207de51c0371b0bd1d4d4cb8cb82dbd6213df04e54e WatchSource:0}: Error finding container 2556b9e916acfda1dcb9d207de51c0371b0bd1d4d4cb8cb82dbd6213df04e54e: Status 404 returned error can't find the container with id 2556b9e916acfda1dcb9d207de51c0371b0bd1d4d4cb8cb82dbd6213df04e54e
Jul 17 06:06:06.658044 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:06.658008    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg" event={"ID":"c03a52bc-8bfe-4543-8bef-0d29efde9b16","Type":"ContainerStarted","Data":"ec0b1343e116b647bc7ee06d6ee1ddf95503884daf29cc821e9a81258a6d2a7b"}
Jul 17 06:06:06.658257 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:06.658058    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg" event={"ID":"c03a52bc-8bfe-4543-8bef-0d29efde9b16","Type":"ContainerStarted","Data":"2556b9e916acfda1dcb9d207de51c0371b0bd1d4d4cb8cb82dbd6213df04e54e"}
Jul 17 06:06:06.658257 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:06.658088    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"
Jul 17 06:06:06.678186 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:06.678140    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg" podStartSLOduration=1.678123365 podStartE2EDuration="1.678123365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:06:06.675574383 +0000 UTC m=+1489.117512960" watchObservedRunningTime="2026-07-17 06:06:06.678123365 +0000 UTC m=+1489.120062005"
Jul 17 06:06:17.665728 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:17.665691    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j6tkg"
Jul 17 06:06:18.045229 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:18.045147    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:06:18.049776 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:18.049753    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:06:18.051035 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:18.051015    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:06:18.055111 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:06:18.055091    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:11:18.078813 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:11:18.078785    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:11:18.085023 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:11:18.084999    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:11:18.085517 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:11:18.085497    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:11:18.091086 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:11:18.091069    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:12:08.961965 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:08.961934    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"]
Jul 17 06:12:08.965550 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:08.965530    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:08.970092 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:08.970074    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0c567a84-openshift-default-dockercfg-nbgr8\""
Jul 17 06:12:09.007935 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.007913    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"]
Jul 17 06:12:09.014916 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.014890    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gldnl\" (UniqueName: \"kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-kube-api-access-gldnl\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.015008 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.014938    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-istio-token\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.015008 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.015002    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-envoy\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.015101 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.015062    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72978a52-902e-4054-aa3a-399029e3df48-istio-podinfo\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.015177 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.015164    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-socket\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.015215 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.015191    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-data\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.015254 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.015230    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-certs\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.015294 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.015265    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-credential-socket\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.015294 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.015284    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72978a52-902e-4054-aa3a-399029e3df48-istiod-ca-cert\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.116615 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.116582    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-certs\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.116615 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.116616    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-credential-socket\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.116856 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.116632    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72978a52-902e-4054-aa3a-399029e3df48-istiod-ca-cert\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.116856 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.116667    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gldnl\" (UniqueName: \"kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-kube-api-access-gldnl\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.116856 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.116815    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-istio-token\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.117027 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.116882    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-envoy\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.117027 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.116920    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72978a52-902e-4054-aa3a-399029e3df48-istio-podinfo\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.117027 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.116953    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-certs\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.117190 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.117019    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-credential-socket\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.117190 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.117023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-socket\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.117190 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.117084    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-data\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.117359 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.117322    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-socket\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.117417 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.117393    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-data\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.117459 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.117396    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72978a52-902e-4054-aa3a-399029e3df48-istiod-ca-cert\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.119285 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.119264    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-envoy\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.119549 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.119531    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72978a52-902e-4054-aa3a-399029e3df48-istio-podinfo\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.131039 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.130998    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-istio-token\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.131158 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.131001    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gldnl\" (UniqueName: \"kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-kube-api-access-gldnl\") pod \"e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") " pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.276013 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.275940    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:09.453250 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.453224    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"]
Jul 17 06:12:09.455558 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:12:09.455533    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72978a52_902e_4054_aa3a_399029e3df48.slice/crio-0f02690e99f95b27a992e8562e05cd9888ba869be4aae9785654277329e7c694 WatchSource:0}: Error finding container 0f02690e99f95b27a992e8562e05cd9888ba869be4aae9785654277329e7c694: Status 404 returned error can't find the container with id 0f02690e99f95b27a992e8562e05cd9888ba869be4aae9785654277329e7c694
Jul 17 06:12:09.457853 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.457806    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:12:09.457992 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.457909    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:12:09.457992 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.457964    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:12:09.516973 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:09.516946    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" event={"ID":"72978a52-902e-4054-aa3a-399029e3df48","Type":"ContainerStarted","Data":"0f02690e99f95b27a992e8562e05cd9888ba869be4aae9785654277329e7c694"}
Jul 17 06:12:10.525296 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:10.525257    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" event={"ID":"72978a52-902e-4054-aa3a-399029e3df48","Type":"ContainerStarted","Data":"47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b"}
Jul 17 06:12:10.594112 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:10.594066    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" podStartSLOduration=2.59405055 podStartE2EDuration="2.59405055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:12:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:12:10.593591871 +0000 UTC m=+1853.035530471" watchObservedRunningTime="2026-07-17 06:12:10.59405055 +0000 UTC m=+1853.035989127"
Jul 17 06:12:11.277230 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:11.277194    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:11.282204 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:11.282179    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:11.532406 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:11.532323    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:11.533148 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:11.533130    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:15.167676 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:15.167639    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"]
Jul 17 06:12:15.171624 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:15.171602    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"
Jul 17 06:12:15.174180 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:15.174154    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:12:15.178117 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:15.178088    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"]
Jul 17 06:12:15.275221 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:15.275181    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w\" (UID: \"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"
Jul 17 06:12:15.375893 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:15.375857    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w\" (UID: \"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"
Jul 17 06:12:15.376519 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:15.376500    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w\" (UID: \"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"
Jul 17 06:12:15.484922 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:15.484843    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"
Jul 17 06:12:15.833297 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:15.833267    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"]
Jul 17 06:12:15.834869 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:12:15.834841    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef4ebfa_ef2f_48ef_bcb2_8fe38d8dc9ca.slice/crio-3ff20a45f8fe1e88526b520244db4c627bee133096537cc202e516ae2445e795 WatchSource:0}: Error finding container 3ff20a45f8fe1e88526b520244db4c627bee133096537cc202e516ae2445e795: Status 404 returned error can't find the container with id 3ff20a45f8fe1e88526b520244db4c627bee133096537cc202e516ae2445e795
Jul 17 06:12:16.563896 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:16.563811    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w" event={"ID":"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca","Type":"ContainerStarted","Data":"5c9b4de3c1484eed630b26f4ab21b127334f75fa487a22028e2724b76e783527"}
Jul 17 06:12:16.563896 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:16.563873    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w" event={"ID":"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca","Type":"ContainerStarted","Data":"3ff20a45f8fe1e88526b520244db4c627bee133096537cc202e516ae2445e795"}
Jul 17 06:12:16.582945 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:16.582896    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w" podStartSLOduration=1.582875969 podStartE2EDuration="1.582875969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:12:16.581581513 +0000 UTC m=+1859.023520088" watchObservedRunningTime="2026-07-17 06:12:16.582875969 +0000 UTC m=+1859.024814544"
Jul 17 06:12:17.570260 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:17.570230    2558 generic.go:356] "Generic (PLEG): container finished" podID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerID="5c9b4de3c1484eed630b26f4ab21b127334f75fa487a22028e2724b76e783527" exitCode=7
Jul 17 06:12:17.570649 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:17.570318    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w" event={"ID":"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca","Type":"ContainerDied","Data":"5c9b4de3c1484eed630b26f4ab21b127334f75fa487a22028e2724b76e783527"}
Jul 17 06:12:17.570649 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:17.570446    2558 scope.go:117] "RemoveContainer" containerID="5c9b4de3c1484eed630b26f4ab21b127334f75fa487a22028e2724b76e783527"
Jul 17 06:12:18.577423 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:18.577389    2558 generic.go:356] "Generic (PLEG): container finished" podID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerID="c457471c974613e535276c627f41baa4c6dac0fcf25d616850254676f46571ae" exitCode=7
Jul 17 06:12:18.577852 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:18.577494    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w" event={"ID":"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca","Type":"ContainerDied","Data":"c457471c974613e535276c627f41baa4c6dac0fcf25d616850254676f46571ae"}
Jul 17 06:12:18.577852 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:18.577554    2558 scope.go:117] "RemoveContainer" containerID="5c9b4de3c1484eed630b26f4ab21b127334f75fa487a22028e2724b76e783527"
Jul 17 06:12:18.577852 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:18.577704    2558 scope.go:117] "RemoveContainer" containerID="c457471c974613e535276c627f41baa4c6dac0fcf25d616850254676f46571ae"
Jul 17 06:12:18.578035 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:12:18.578004    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w_odh-ai-gateway-infra(7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca"
Jul 17 06:12:19.584532 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:19.584501    2558 scope.go:117] "RemoveContainer" containerID="c457471c974613e535276c627f41baa4c6dac0fcf25d616850254676f46571ae"
Jul 17 06:12:19.584909 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:12:19.584715    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w_odh-ai-gateway-infra(7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca"
Jul 17 06:12:26.534340 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:26.534301    2558 patch_prober.go:28] interesting pod/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 17 06:12:26.534745 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:26.534376    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 17 06:12:26.621599 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:26.621573    2558 generic.go:356] "Generic (PLEG): container finished" podID="72978a52-902e-4054-aa3a-399029e3df48" containerID="47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b" exitCode=0
Jul 17 06:12:26.621785 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:26.621660    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" event={"ID":"72978a52-902e-4054-aa3a-399029e3df48","Type":"ContainerDied","Data":"47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b"}
Jul 17 06:12:26.622045 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:26.622030    2558 scope.go:117] "RemoveContainer" containerID="47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b"
Jul 17 06:12:26.623189 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:26.623151    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:12:26.623283 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:26.623245    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:12:26.623322 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:26.623293    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:12:27.633436 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:27.632451    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" event={"ID":"72978a52-902e-4054-aa3a-399029e3df48","Type":"ContainerStarted","Data":"d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342"}
Jul 17 06:12:28.276887 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:28.276850    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:28.277097 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:28.277033    2558 patch_prober.go:28] interesting pod/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 17 06:12:28.277178 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:28.277088    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 17 06:12:29.276865 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:29.276824    2558 patch_prober.go:28] interesting pod/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 17 06:12:29.277253 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:29.276891    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 17 06:12:30.072488 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:30.072442    2558 scope.go:117] "RemoveContainer" containerID="c457471c974613e535276c627f41baa4c6dac0fcf25d616850254676f46571ae"
Jul 17 06:12:30.276737 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:30.276707    2558 patch_prober.go:28] interesting pod/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 17 06:12:30.276861 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:30.276760    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 17 06:12:30.657622 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:30.657588    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w" event={"ID":"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca","Type":"ContainerStarted","Data":"ab028aaf3854032fc4c86bdf2ac170b9c0f2e834d101b92e8d1c563b1db6bfac"}
Jul 17 06:12:31.276980 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:31.276884    2558 patch_prober.go:28] interesting pod/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 17 06:12:31.276980 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:31.276951    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 17 06:12:31.664403 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:31.664370    2558 generic.go:356] "Generic (PLEG): container finished" podID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerID="ab028aaf3854032fc4c86bdf2ac170b9c0f2e834d101b92e8d1c563b1db6bfac" exitCode=7
Jul 17 06:12:31.664829 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:31.664442    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w" event={"ID":"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca","Type":"ContainerDied","Data":"ab028aaf3854032fc4c86bdf2ac170b9c0f2e834d101b92e8d1c563b1db6bfac"}
Jul 17 06:12:31.664829 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:31.664530    2558 scope.go:117] "RemoveContainer" containerID="c457471c974613e535276c627f41baa4c6dac0fcf25d616850254676f46571ae"
Jul 17 06:12:31.664952 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:31.664844    2558 scope.go:117] "RemoveContainer" containerID="ab028aaf3854032fc4c86bdf2ac170b9c0f2e834d101b92e8d1c563b1db6bfac"
Jul 17 06:12:31.665053 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:12:31.665032    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w_odh-ai-gateway-infra(7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca"
Jul 17 06:12:31.702673 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:31.702643    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"]
Jul 17 06:12:32.280933 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:32.280906    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:32.281261 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:32.281245    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:32.281824 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:32.281807    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:12:32.814695 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:32.814669    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"
Jul 17 06:12:32.934345 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:32.934261    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca-maas-api-service-ca\") pod \"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca\" (UID: \"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca\") "
Jul 17 06:12:32.934644 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:32.934624    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" (UID: "7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:12:33.035152 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:33.035124    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:12:33.678124 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:33.678088    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"
Jul 17 06:12:33.678305 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:33.678130    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w" event={"ID":"7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca","Type":"ContainerDied","Data":"3ff20a45f8fe1e88526b520244db4c627bee133096537cc202e516ae2445e795"}
Jul 17 06:12:33.678305 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:33.678187    2558 scope.go:117] "RemoveContainer" containerID="ab028aaf3854032fc4c86bdf2ac170b9c0f2e834d101b92e8d1c563b1db6bfac"
Jul 17 06:12:33.700825 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:33.700803    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"]
Jul 17 06:12:33.706981 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:33.706959    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-nqv9w"]
Jul 17 06:12:34.079708 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:12:34.079629    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" path="/var/lib/kubelet/pods/7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca/volumes"
Jul 17 06:15:00.138134 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.138102    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737815-bfzf4"]
Jul 17 06:15:00.138630 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.138494    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerName="revoke-keys"
Jul 17 06:15:00.138630 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.138505    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerName="revoke-keys"
Jul 17 06:15:00.138630 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.138528    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerName="revoke-keys"
Jul 17 06:15:00.138630 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.138533    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerName="revoke-keys"
Jul 17 06:15:00.138630 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.138594    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerName="revoke-keys"
Jul 17 06:15:00.138630 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.138603    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerName="revoke-keys"
Jul 17 06:15:00.140631 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.140614    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737815-bfzf4"
Jul 17 06:15:00.143116 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.143091    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:15:00.148767 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.148740    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737815-bfzf4"]
Jul 17 06:15:00.151733 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.151711    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737815-bfzf4"
Jul 17 06:15:00.288294 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.288270    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737815-bfzf4"]
Jul 17 06:15:00.290563 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:15:00.290533    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98e314d1_895c_4b9f_a58b_e69257fe4851.slice/crio-2c832772df1492ccd2fca4593b720e9cda42b12299e0953449e0ef55d38f21fa WatchSource:0}: Error finding container 2c832772df1492ccd2fca4593b720e9cda42b12299e0953449e0ef55d38f21fa: Status 404 returned error can't find the container with id 2c832772df1492ccd2fca4593b720e9cda42b12299e0953449e0ef55d38f21fa
Jul 17 06:15:00.458013 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:00.457934    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737815-bfzf4" event={"ID":"98e314d1-895c-4b9f-a58b-e69257fe4851","Type":"ContainerStarted","Data":"2c832772df1492ccd2fca4593b720e9cda42b12299e0953449e0ef55d38f21fa"}
Jul 17 06:15:01.469692 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:01.469648    2558 generic.go:356] "Generic (PLEG): container finished" podID="98e314d1-895c-4b9f-a58b-e69257fe4851" containerID="1cba45ba73cd03a11ea3d38e9bca45230ef37d4e401f4abbba383bd20ee8bee3" exitCode=0
Jul 17 06:15:01.469692 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:01.469684    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737815-bfzf4" event={"ID":"98e314d1-895c-4b9f-a58b-e69257fe4851","Type":"ContainerDied","Data":"1cba45ba73cd03a11ea3d38e9bca45230ef37d4e401f4abbba383bd20ee8bee3"}
Jul 17 06:15:02.615186 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:02.615157    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737815-bfzf4"
Jul 17 06:15:03.482414 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:03.482384    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737815-bfzf4"
Jul 17 06:15:03.482414 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:03.482407    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737815-bfzf4" event={"ID":"98e314d1-895c-4b9f-a58b-e69257fe4851","Type":"ContainerDied","Data":"2c832772df1492ccd2fca4593b720e9cda42b12299e0953449e0ef55d38f21fa"}
Jul 17 06:15:03.482659 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:03.482451    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c832772df1492ccd2fca4593b720e9cda42b12299e0953449e0ef55d38f21fa"
Jul 17 06:15:15.299904 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:15.299863    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"]
Jul 17 06:15:15.300364 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:15.300094    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy" containerID="cri-o://d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342" gracePeriod=30
Jul 17 06:15:17.283286 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:17.283251    2558 patch_prober.go:28] interesting pod/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 06:15:17.283688 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:17.283315    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 06:15:20.564174 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.564118    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:15:20.566920 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.566896    2558 generic.go:356] "Generic (PLEG): container finished" podID="72978a52-902e-4054-aa3a-399029e3df48" containerID="d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342" exitCode=0
Jul 17 06:15:20.567034 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.566931    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"
Jul 17 06:15:20.567034 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.566965    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" event={"ID":"72978a52-902e-4054-aa3a-399029e3df48","Type":"ContainerDied","Data":"d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342"}
Jul 17 06:15:20.567034 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.567011    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq" event={"ID":"72978a52-902e-4054-aa3a-399029e3df48","Type":"ContainerDied","Data":"0f02690e99f95b27a992e8562e05cd9888ba869be4aae9785654277329e7c694"}
Jul 17 06:15:20.567034 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.567033    2558 scope.go:117] "RemoveContainer" containerID="d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342"
Jul 17 06:15:20.575702 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.575681    2558 scope.go:117] "RemoveContainer" containerID="47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b"
Jul 17 06:15:20.584044 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.584000    2558 scope.go:117] "RemoveContainer" containerID="d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342"
Jul 17 06:15:20.584409 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:15:20.584376    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342\": container with ID starting with d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342 not found: ID does not exist" containerID="d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342"
Jul 17 06:15:20.584545 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.584418    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342"} err="failed to get container status \"d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342\": rpc error: code = NotFound desc = could not find container \"d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342\": container with ID starting with d33b82c2212eedfc55cb4bb4b93835eda3a0304f7abf645fb45990540dab8342 not found: ID does not exist"
Jul 17 06:15:20.584545 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.584442    2558 scope.go:117] "RemoveContainer" containerID="47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b"
Jul 17 06:15:20.584853 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:15:20.584831    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b\": container with ID starting with 47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b not found: ID does not exist" containerID="47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b"
Jul 17 06:15:20.584934 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.584863    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b"} err="failed to get container status \"47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b\": rpc error: code = NotFound desc = could not find container \"47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b\": container with ID starting with 47236843633a9001171529512a61b17997a4741109727c5bb6434a951bd48e8b not found: ID does not exist"
Jul 17 06:15:20.591790 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.591764    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-istio-token\") pod \"72978a52-902e-4054-aa3a-399029e3df48\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") "
Jul 17 06:15:20.591897 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.591807    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72978a52-902e-4054-aa3a-399029e3df48-istio-podinfo\") pod \"72978a52-902e-4054-aa3a-399029e3df48\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") "
Jul 17 06:15:20.591897 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.591867    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-socket\") pod \"72978a52-902e-4054-aa3a-399029e3df48\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") "
Jul 17 06:15:20.592013 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.591904    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-envoy\") pod \"72978a52-902e-4054-aa3a-399029e3df48\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") "
Jul 17 06:15:20.592013 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.591928    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72978a52-902e-4054-aa3a-399029e3df48-istiod-ca-cert\") pod \"72978a52-902e-4054-aa3a-399029e3df48\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") "
Jul 17 06:15:20.592013 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.592008    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-data\") pod \"72978a52-902e-4054-aa3a-399029e3df48\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") "
Jul 17 06:15:20.592159 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.592041    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-credential-socket\") pod \"72978a52-902e-4054-aa3a-399029e3df48\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") "
Jul 17 06:15:20.592159 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.592097    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gldnl\" (UniqueName: \"kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-kube-api-access-gldnl\") pod \"72978a52-902e-4054-aa3a-399029e3df48\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") "
Jul 17 06:15:20.592159 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.592132    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-certs\") pod \"72978a52-902e-4054-aa3a-399029e3df48\" (UID: \"72978a52-902e-4054-aa3a-399029e3df48\") "
Jul 17 06:15:20.593727 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.592674    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72978a52-902e-4054-aa3a-399029e3df48-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "72978a52-902e-4054-aa3a-399029e3df48" (UID: "72978a52-902e-4054-aa3a-399029e3df48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:15:20.593727 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.592677    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "72978a52-902e-4054-aa3a-399029e3df48" (UID: "72978a52-902e-4054-aa3a-399029e3df48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:15:20.593727 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.592686    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "72978a52-902e-4054-aa3a-399029e3df48" (UID: "72978a52-902e-4054-aa3a-399029e3df48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:15:20.593727 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.593693    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-data" (OuterVolumeSpecName: "istio-data") pod "72978a52-902e-4054-aa3a-399029e3df48" (UID: "72978a52-902e-4054-aa3a-399029e3df48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:15:20.594157 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.594128    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "72978a52-902e-4054-aa3a-399029e3df48" (UID: "72978a52-902e-4054-aa3a-399029e3df48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:15:20.594772 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.594739    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-istio-token" (OuterVolumeSpecName: "istio-token") pod "72978a52-902e-4054-aa3a-399029e3df48" (UID: "72978a52-902e-4054-aa3a-399029e3df48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:15:20.595219 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.595192    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-kube-api-access-gldnl" (OuterVolumeSpecName: "kube-api-access-gldnl") pod "72978a52-902e-4054-aa3a-399029e3df48" (UID: "72978a52-902e-4054-aa3a-399029e3df48"). InnerVolumeSpecName "kube-api-access-gldnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:15:20.595461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.595442    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "72978a52-902e-4054-aa3a-399029e3df48" (UID: "72978a52-902e-4054-aa3a-399029e3df48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:15:20.596164 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.596145    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72978a52-902e-4054-aa3a-399029e3df48-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "72978a52-902e-4054-aa3a-399029e3df48" (UID: "72978a52-902e-4054-aa3a-399029e3df48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:15:20.692859 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.692831    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:15:20.692859 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.692859    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:15:20.693008 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.692867    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72978a52-902e-4054-aa3a-399029e3df48-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:15:20.693008 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.692875    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:15:20.693008 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.692882    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:15:20.693008 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.692890    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72978a52-902e-4054-aa3a-399029e3df48-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:15:20.693008 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.692897    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:15:20.693008 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.692905    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72978a52-902e-4054-aa3a-399029e3df48-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:15:20.693008 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.692913    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gldnl\" (UniqueName: \"kubernetes.io/projected/72978a52-902e-4054-aa3a-399029e3df48-kube-api-access-gldnl\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:15:20.894966 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.894938    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"]
Jul 17 06:15:20.900031 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:20.900007    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0c567a84-openshift-default-7469f49587-nt4wq"]
Jul 17 06:15:22.078667 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:22.078631    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72978a52-902e-4054-aa3a-399029e3df48" path="/var/lib/kubelet/pods/72978a52-902e-4054-aa3a-399029e3df48/volumes"
Jul 17 06:15:27.397795 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.397760    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"]
Jul 17 06:15:27.398805 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.398779    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy"
Jul 17 06:15:27.398805 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.398802    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy"
Jul 17 06:15:27.399014 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.398813    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98e314d1-895c-4b9f-a58b-e69257fe4851" containerName="cleanup"
Jul 17 06:15:27.399014 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.398819    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="98e314d1-895c-4b9f-a58b-e69257fe4851" containerName="cleanup"
Jul 17 06:15:27.399014 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.398837    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerName="revoke-keys"
Jul 17 06:15:27.399014 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.398841    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerName="revoke-keys"
Jul 17 06:15:27.399014 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.398904    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ef4ebfa-ef2f-48ef-bcb2-8fe38d8dc9ca" containerName="revoke-keys"
Jul 17 06:15:27.399014 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.398913    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy"
Jul 17 06:15:27.399014 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.398919    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy"
Jul 17 06:15:27.399014 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.398927    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="98e314d1-895c-4b9f-a58b-e69257fe4851" containerName="cleanup"
Jul 17 06:15:27.403684 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.403665    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.406551 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.406531    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-a9646996-gw-openshift-default-dockercfg-tqmv4\""
Jul 17 06:15:27.414770 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.414748    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"]
Jul 17 06:15:27.447093 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.447064    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-credential-socket\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.447207 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.447097    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-socket\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.447207 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.447118    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-token\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.447329 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.447233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-podinfo\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.447329 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.447302    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-data\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.447434 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.447335    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbrkw\" (UniqueName: \"kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-kube-api-access-zbrkw\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.447434 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.447404    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-envoy\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.447565 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.447490    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/877a6520-a5ba-45f6-932e-6b47d170a6a0-istiod-ca-cert\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.447565 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.447520    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-certs\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.548555 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.548522    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-podinfo\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.548719 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.548570    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-data\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.548719 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.548602    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbrkw\" (UniqueName: \"kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-kube-api-access-zbrkw\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.548719 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.548645    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-envoy\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.548719 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.548693    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/877a6520-a5ba-45f6-932e-6b47d170a6a0-istiod-ca-cert\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.548941 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.548720    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-certs\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.548941 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.548805    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-credential-socket\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.548941 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.548835    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-socket\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.548941 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.548859    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-token\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.549147 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.549115    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-data\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.549244 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.549213    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-certs\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.549585 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.549264    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-credential-socket\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.549585 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.549419    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-socket\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.549722 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.549689    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/877a6520-a5ba-45f6-932e-6b47d170a6a0-istiod-ca-cert\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.551116 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.551091    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-envoy\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.551267 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.551251    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-podinfo\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.557750 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.557725    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-token\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.557832 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.557727    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbrkw\" (UniqueName: \"kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-kube-api-access-zbrkw\") pod \"e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") " pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.714752 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.714682    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:27.869209 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.869180    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"]
Jul 17 06:15:27.871181 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:15:27.871157    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877a6520_a5ba_45f6_932e_6b47d170a6a0.slice/crio-cecc551ff539b3dd985aac7a86b44213e53dfa22b552df22509b810fafb06ea0 WatchSource:0}: Error finding container cecc551ff539b3dd985aac7a86b44213e53dfa22b552df22509b810fafb06ea0: Status 404 returned error can't find the container with id cecc551ff539b3dd985aac7a86b44213e53dfa22b552df22509b810fafb06ea0
Jul 17 06:15:27.873319 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.873283    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:15:27.873394 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.873375    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:15:27.873439 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:27.873423    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:15:28.608876 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:28.608843    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w" event={"ID":"877a6520-a5ba-45f6-932e-6b47d170a6a0","Type":"ContainerStarted","Data":"621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2"}
Jul 17 06:15:28.609243 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:28.608902    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w" event={"ID":"877a6520-a5ba-45f6-932e-6b47d170a6a0","Type":"ContainerStarted","Data":"cecc551ff539b3dd985aac7a86b44213e53dfa22b552df22509b810fafb06ea0"}
Jul 17 06:15:28.634624 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:28.634576    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w" podStartSLOduration=1.63456102 podStartE2EDuration="1.63456102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:15:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:15:28.632565138 +0000 UTC m=+2051.074503709" watchObservedRunningTime="2026-07-17 06:15:28.63456102 +0000 UTC m=+2051.076499607"
Jul 17 06:15:28.714978 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:28.714950    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:28.720698 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:28.720673    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:29.614626 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:29.614599    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:29.615562 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:29.615543    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:33.200740 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:33.200708    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"]
Jul 17 06:15:33.201320 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:33.201299    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy"
Jul 17 06:15:33.201408 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:33.201323    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="72978a52-902e-4054-aa3a-399029e3df48" containerName="istio-proxy"
Jul 17 06:15:33.205616 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:33.205594    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"
Jul 17 06:15:33.208611 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:33.208587    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:15:33.210777 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:33.210755    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"]
Jul 17 06:15:33.307884 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:33.307855    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4c7b87-01e1-42b4-873f-a822639eea19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv\" (UID: \"5b4c7b87-01e1-42b4-873f-a822639eea19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"
Jul 17 06:15:33.408543 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:33.408510    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4c7b87-01e1-42b4-873f-a822639eea19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv\" (UID: \"5b4c7b87-01e1-42b4-873f-a822639eea19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"
Jul 17 06:15:33.409160 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:33.409138    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4c7b87-01e1-42b4-873f-a822639eea19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv\" (UID: \"5b4c7b87-01e1-42b4-873f-a822639eea19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"
Jul 17 06:15:33.518871 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:33.518805    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"
Jul 17 06:15:33.704947 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:33.704919    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"]
Jul 17 06:15:33.708210 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:15:33.708179    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4c7b87_01e1_42b4_873f_a822639eea19.slice/crio-031c9547a920644e9ed67ef92b00b446c12977ea0ce992623478d540ede8f79e WatchSource:0}: Error finding container 031c9547a920644e9ed67ef92b00b446c12977ea0ce992623478d540ede8f79e: Status 404 returned error can't find the container with id 031c9547a920644e9ed67ef92b00b446c12977ea0ce992623478d540ede8f79e
Jul 17 06:15:34.674075 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:34.674044    2558 generic.go:356] "Generic (PLEG): container finished" podID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerID="5216a966d5838e262ebbafd3798c02b2ec49079b0bf8a1ec0022691dffa94088" exitCode=7
Jul 17 06:15:34.674454 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:34.674131    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv" event={"ID":"5b4c7b87-01e1-42b4-873f-a822639eea19","Type":"ContainerDied","Data":"5216a966d5838e262ebbafd3798c02b2ec49079b0bf8a1ec0022691dffa94088"}
Jul 17 06:15:34.674454 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:34.674185    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv" event={"ID":"5b4c7b87-01e1-42b4-873f-a822639eea19","Type":"ContainerStarted","Data":"031c9547a920644e9ed67ef92b00b446c12977ea0ce992623478d540ede8f79e"}
Jul 17 06:15:34.674454 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:34.674295    2558 scope.go:117] "RemoveContainer" containerID="5216a966d5838e262ebbafd3798c02b2ec49079b0bf8a1ec0022691dffa94088"
Jul 17 06:15:35.682670 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:35.682455    2558 generic.go:356] "Generic (PLEG): container finished" podID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerID="1d4c075ef78389c51226780f35123701f22df4efcccd98c336394d2165f2ed45" exitCode=7
Jul 17 06:15:35.682670 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:35.682520    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv" event={"ID":"5b4c7b87-01e1-42b4-873f-a822639eea19","Type":"ContainerDied","Data":"1d4c075ef78389c51226780f35123701f22df4efcccd98c336394d2165f2ed45"}
Jul 17 06:15:35.682670 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:35.682605    2558 scope.go:117] "RemoveContainer" containerID="5216a966d5838e262ebbafd3798c02b2ec49079b0bf8a1ec0022691dffa94088"
Jul 17 06:15:35.683211 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:35.682818    2558 scope.go:117] "RemoveContainer" containerID="1d4c075ef78389c51226780f35123701f22df4efcccd98c336394d2165f2ed45"
Jul 17 06:15:35.683211 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:15:35.683052    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv_odh-ai-gateway-infra(5b4c7b87-01e1-42b4-873f-a822639eea19)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19"
Jul 17 06:15:36.690212 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:36.690177    2558 scope.go:117] "RemoveContainer" containerID="1d4c075ef78389c51226780f35123701f22df4efcccd98c336394d2165f2ed45"
Jul 17 06:15:36.690699 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:15:36.690464    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv_odh-ai-gateway-infra(5b4c7b87-01e1-42b4-873f-a822639eea19)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19"
Jul 17 06:15:39.708134 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:39.708098    2558 generic.go:356] "Generic (PLEG): container finished" podID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerID="621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2" exitCode=0
Jul 17 06:15:39.708543 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:39.708170    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w" event={"ID":"877a6520-a5ba-45f6-932e-6b47d170a6a0","Type":"ContainerDied","Data":"621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2"}
Jul 17 06:15:39.708543 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:39.708534    2558 scope.go:117] "RemoveContainer" containerID="621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2"
Jul 17 06:15:39.709511 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:39.709463    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:15:39.709570 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:39.709556    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:15:39.709606 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:39.709596    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:15:40.715541 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:40.715500    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w" event={"ID":"877a6520-a5ba-45f6-932e-6b47d170a6a0","Type":"ContainerStarted","Data":"1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc"}
Jul 17 06:15:41.714969 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:41.714934    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:41.716201 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:41.716175    2558 patch_prober.go:28] interesting pod/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 17 06:15:41.716543 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:41.716227    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w" podUID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 17 06:15:42.715580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:42.715547    2558 patch_prober.go:28] interesting pod/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 17 06:15:42.715753 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:42.715599    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w" podUID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 17 06:15:43.715980 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:43.715942    2558 patch_prober.go:28] interesting pod/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 17 06:15:43.716382 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:43.716007    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w" podUID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 17 06:15:44.615174 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:44.615143    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:44.719107 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:44.719083    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:44.755411 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:44.755385    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:15:51.072725 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:51.072697    2558 scope.go:117] "RemoveContainer" containerID="1d4c075ef78389c51226780f35123701f22df4efcccd98c336394d2165f2ed45"
Jul 17 06:15:51.791863 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:51.791839    2558 generic.go:356] "Generic (PLEG): container finished" podID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerID="cf8e297b2504472922223a04f690d033c86305cb78354adf8d2467b4f776a8cc" exitCode=7
Jul 17 06:15:51.792000 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:51.791897    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv" event={"ID":"5b4c7b87-01e1-42b4-873f-a822639eea19","Type":"ContainerDied","Data":"cf8e297b2504472922223a04f690d033c86305cb78354adf8d2467b4f776a8cc"}
Jul 17 06:15:51.792000 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:51.791958    2558 scope.go:117] "RemoveContainer" containerID="1d4c075ef78389c51226780f35123701f22df4efcccd98c336394d2165f2ed45"
Jul 17 06:15:51.792302 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:51.792280    2558 scope.go:117] "RemoveContainer" containerID="cf8e297b2504472922223a04f690d033c86305cb78354adf8d2467b4f776a8cc"
Jul 17 06:15:51.792528 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:15:51.792507    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv_odh-ai-gateway-infra(5b4c7b87-01e1-42b4-873f-a822639eea19)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19"
Jul 17 06:15:52.817858 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:52.817823    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"]
Jul 17 06:15:52.958029 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:52.958006    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"
Jul 17 06:15:53.082360 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:53.082337    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4c7b87-01e1-42b4-873f-a822639eea19-maas-api-service-ca\") pod \"5b4c7b87-01e1-42b4-873f-a822639eea19\" (UID: \"5b4c7b87-01e1-42b4-873f-a822639eea19\") "
Jul 17 06:15:53.082659 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:53.082638    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b4c7b87-01e1-42b4-873f-a822639eea19-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5b4c7b87-01e1-42b4-873f-a822639eea19" (UID: "5b4c7b87-01e1-42b4-873f-a822639eea19"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:15:53.183487 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:53.183446    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4c7b87-01e1-42b4-873f-a822639eea19-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:15:53.805988 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:53.805961    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"
Jul 17 06:15:53.806166 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:53.805998    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv" event={"ID":"5b4c7b87-01e1-42b4-873f-a822639eea19","Type":"ContainerDied","Data":"031c9547a920644e9ed67ef92b00b446c12977ea0ce992623478d540ede8f79e"}
Jul 17 06:15:53.806166 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:53.806059    2558 scope.go:117] "RemoveContainer" containerID="cf8e297b2504472922223a04f690d033c86305cb78354adf8d2467b4f776a8cc"
Jul 17 06:15:53.829212 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:53.829187    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"]
Jul 17 06:15:53.835734 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:53.835713    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-n2vgv"]
Jul 17 06:15:54.078308 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:54.078244    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19" path="/var/lib/kubelet/pods/5b4c7b87-01e1-42b4-873f-a822639eea19/volumes"
Jul 17 06:15:55.345021 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.344990    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"]
Jul 17 06:15:55.345386 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.345374    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerName="revoke-keys"
Jul 17 06:15:55.345429 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.345388    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerName="revoke-keys"
Jul 17 06:15:55.345429 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.345395    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerName="revoke-keys"
Jul 17 06:15:55.345429 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.345400    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerName="revoke-keys"
Jul 17 06:15:55.345553 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.345493    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerName="revoke-keys"
Jul 17 06:15:55.345553 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.345504    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerName="revoke-keys"
Jul 17 06:15:55.345553 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.345510    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerName="revoke-keys"
Jul 17 06:15:55.348363 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.348347    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"
Jul 17 06:15:55.350926 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.350906    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:15:55.355388 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.355362    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"]
Jul 17 06:15:55.498871 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.498844    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9a71cd-7afc-4111-af26-f0342d9e8d51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-lvnql\" (UID: \"8a9a71cd-7afc-4111-af26-f0342d9e8d51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"
Jul 17 06:15:55.599986 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.599910    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9a71cd-7afc-4111-af26-f0342d9e8d51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-lvnql\" (UID: \"8a9a71cd-7afc-4111-af26-f0342d9e8d51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"
Jul 17 06:15:55.600490 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.600450    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9a71cd-7afc-4111-af26-f0342d9e8d51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-lvnql\" (UID: \"8a9a71cd-7afc-4111-af26-f0342d9e8d51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"
Jul 17 06:15:55.660446 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.660423    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"
Jul 17 06:15:55.788178 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.788154    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"]
Jul 17 06:15:55.789659 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:15:55.789634    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a9a71cd_7afc_4111_af26_f0342d9e8d51.slice/crio-11aa8aa488852017ed817629df12ea05effbbfee04019a68a8f21525bca8018a WatchSource:0}: Error finding container 11aa8aa488852017ed817629df12ea05effbbfee04019a68a8f21525bca8018a: Status 404 returned error can't find the container with id 11aa8aa488852017ed817629df12ea05effbbfee04019a68a8f21525bca8018a
Jul 17 06:15:55.818391 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:55.818362    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql" event={"ID":"8a9a71cd-7afc-4111-af26-f0342d9e8d51","Type":"ContainerStarted","Data":"11aa8aa488852017ed817629df12ea05effbbfee04019a68a8f21525bca8018a"}
Jul 17 06:15:56.824889 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:56.824813    2558 generic.go:356] "Generic (PLEG): container finished" podID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerID="8c48136fe5c216fd17fb05fce5ab8c6038a08da089c10bdb2f8d86918577a729" exitCode=6
Jul 17 06:15:56.825241 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:56.824895    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql" event={"ID":"8a9a71cd-7afc-4111-af26-f0342d9e8d51","Type":"ContainerDied","Data":"8c48136fe5c216fd17fb05fce5ab8c6038a08da089c10bdb2f8d86918577a729"}
Jul 17 06:15:56.825241 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:56.825060    2558 scope.go:117] "RemoveContainer" containerID="8c48136fe5c216fd17fb05fce5ab8c6038a08da089c10bdb2f8d86918577a729"
Jul 17 06:15:57.831904 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:57.831876    2558 generic.go:356] "Generic (PLEG): container finished" podID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerID="784acde5f601d9fa6586ae85c104c730140c5fb33aae0c7051cf222d555bdf9c" exitCode=6
Jul 17 06:15:57.832321 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:57.831924    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql" event={"ID":"8a9a71cd-7afc-4111-af26-f0342d9e8d51","Type":"ContainerDied","Data":"784acde5f601d9fa6586ae85c104c730140c5fb33aae0c7051cf222d555bdf9c"}
Jul 17 06:15:57.832321 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:57.831970    2558 scope.go:117] "RemoveContainer" containerID="8c48136fe5c216fd17fb05fce5ab8c6038a08da089c10bdb2f8d86918577a729"
Jul 17 06:15:57.832321 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:57.832178    2558 scope.go:117] "RemoveContainer" containerID="784acde5f601d9fa6586ae85c104c730140c5fb33aae0c7051cf222d555bdf9c"
Jul 17 06:15:57.832514 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:15:57.832382    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-lvnql_odh-ai-gateway-infra(8a9a71cd-7afc-4111-af26-f0342d9e8d51)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51"
Jul 17 06:15:58.838576 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:15:58.838550    2558 scope.go:117] "RemoveContainer" containerID="784acde5f601d9fa6586ae85c104c730140c5fb33aae0c7051cf222d555bdf9c"
Jul 17 06:15:58.838954 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:15:58.838699    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-lvnql_odh-ai-gateway-infra(8a9a71cd-7afc-4111-af26-f0342d9e8d51)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51"
Jul 17 06:16:11.072943 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:11.072915    2558 scope.go:117] "RemoveContainer" containerID="784acde5f601d9fa6586ae85c104c730140c5fb33aae0c7051cf222d555bdf9c"
Jul 17 06:16:11.908022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:11.907943    2558 generic.go:356] "Generic (PLEG): container finished" podID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerID="75ec4a5a2a24d19689cfca736d857cc980351e6cd5e29b8b7143cd6cf0ac6fe4" exitCode=6
Jul 17 06:16:11.908022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:11.907982    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql" event={"ID":"8a9a71cd-7afc-4111-af26-f0342d9e8d51","Type":"ContainerDied","Data":"75ec4a5a2a24d19689cfca736d857cc980351e6cd5e29b8b7143cd6cf0ac6fe4"}
Jul 17 06:16:11.908194 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:11.908036    2558 scope.go:117] "RemoveContainer" containerID="784acde5f601d9fa6586ae85c104c730140c5fb33aae0c7051cf222d555bdf9c"
Jul 17 06:16:11.908329 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:11.908310    2558 scope.go:117] "RemoveContainer" containerID="75ec4a5a2a24d19689cfca736d857cc980351e6cd5e29b8b7143cd6cf0ac6fe4"
Jul 17 06:16:11.908586 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:16:11.908566    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-lvnql_odh-ai-gateway-infra(8a9a71cd-7afc-4111-af26-f0342d9e8d51)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51"
Jul 17 06:16:12.936851 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:12.936811    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"]
Jul 17 06:16:13.084338 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:13.084316    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"
Jul 17 06:16:13.135578 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:13.135555    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9a71cd-7afc-4111-af26-f0342d9e8d51-maas-api-service-ca\") pod \"8a9a71cd-7afc-4111-af26-f0342d9e8d51\" (UID: \"8a9a71cd-7afc-4111-af26-f0342d9e8d51\") "
Jul 17 06:16:13.135905 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:13.135888    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a9a71cd-7afc-4111-af26-f0342d9e8d51-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8a9a71cd-7afc-4111-af26-f0342d9e8d51" (UID: "8a9a71cd-7afc-4111-af26-f0342d9e8d51"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:16:13.236939 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:13.236885    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9a71cd-7afc-4111-af26-f0342d9e8d51-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:16:13.921296 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:13.921270    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"
Jul 17 06:16:13.921460 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:13.921300    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql" event={"ID":"8a9a71cd-7afc-4111-af26-f0342d9e8d51","Type":"ContainerDied","Data":"11aa8aa488852017ed817629df12ea05effbbfee04019a68a8f21525bca8018a"}
Jul 17 06:16:13.921460 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:13.921353    2558 scope.go:117] "RemoveContainer" containerID="75ec4a5a2a24d19689cfca736d857cc980351e6cd5e29b8b7143cd6cf0ac6fe4"
Jul 17 06:16:13.944721 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:13.944700    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"]
Jul 17 06:16:13.952571 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:13.952550    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-lvnql"]
Jul 17 06:16:14.078557 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:14.078530    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" path="/var/lib/kubelet/pods/8a9a71cd-7afc-4111-af26-f0342d9e8d51/volumes"
Jul 17 06:16:18.127074 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:18.127051    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:16:18.130464 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:18.130443    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:16:18.132419 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:18.132401    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:16:18.135880 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:18.135856    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:16:33.312641 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:33.312602    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"]
Jul 17 06:16:33.313160 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:33.312874    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w" podUID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerName="istio-proxy" containerID="cri-o://1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc" gracePeriod=30
Jul 17 06:16:38.579455 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.579390    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:16:38.662335 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662303    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-socket\") pod \"877a6520-a5ba-45f6-932e-6b47d170a6a0\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") "
Jul 17 06:16:38.662531 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662354    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/877a6520-a5ba-45f6-932e-6b47d170a6a0-istiod-ca-cert\") pod \"877a6520-a5ba-45f6-932e-6b47d170a6a0\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") "
Jul 17 06:16:38.662531 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662370    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-certs\") pod \"877a6520-a5ba-45f6-932e-6b47d170a6a0\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") "
Jul 17 06:16:38.662531 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662423    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-credential-socket\") pod \"877a6520-a5ba-45f6-932e-6b47d170a6a0\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") "
Jul 17 06:16:38.662723 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662644    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "877a6520-a5ba-45f6-932e-6b47d170a6a0" (UID: "877a6520-a5ba-45f6-932e-6b47d170a6a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:16:38.662723 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662661    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "877a6520-a5ba-45f6-932e-6b47d170a6a0" (UID: "877a6520-a5ba-45f6-932e-6b47d170a6a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:16:38.662723 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662447    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-podinfo\") pod \"877a6520-a5ba-45f6-932e-6b47d170a6a0\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") "
Jul 17 06:16:38.662897 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662745    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbrkw\" (UniqueName: \"kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-kube-api-access-zbrkw\") pod \"877a6520-a5ba-45f6-932e-6b47d170a6a0\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") "
Jul 17 06:16:38.662897 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662746    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/877a6520-a5ba-45f6-932e-6b47d170a6a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "877a6520-a5ba-45f6-932e-6b47d170a6a0" (UID: "877a6520-a5ba-45f6-932e-6b47d170a6a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:16:38.662897 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662813    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-token\") pod \"877a6520-a5ba-45f6-932e-6b47d170a6a0\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") "
Jul 17 06:16:38.662897 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662837    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "877a6520-a5ba-45f6-932e-6b47d170a6a0" (UID: "877a6520-a5ba-45f6-932e-6b47d170a6a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:16:38.662897 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662841    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-envoy\") pod \"877a6520-a5ba-45f6-932e-6b47d170a6a0\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") "
Jul 17 06:16:38.662897 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.662892    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-data\") pod \"877a6520-a5ba-45f6-932e-6b47d170a6a0\" (UID: \"877a6520-a5ba-45f6-932e-6b47d170a6a0\") "
Jul 17 06:16:38.663339 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.663268    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:16:38.663339 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.663331    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/877a6520-a5ba-45f6-932e-6b47d170a6a0-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:16:38.663487 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.663345    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:16:38.663487 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.663359    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:16:38.664275 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.664243    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "877a6520-a5ba-45f6-932e-6b47d170a6a0" (UID: "877a6520-a5ba-45f6-932e-6b47d170a6a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:16:38.665229 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.665199    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "877a6520-a5ba-45f6-932e-6b47d170a6a0" (UID: "877a6520-a5ba-45f6-932e-6b47d170a6a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:16:38.665340 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.665291    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "877a6520-a5ba-45f6-932e-6b47d170a6a0" (UID: "877a6520-a5ba-45f6-932e-6b47d170a6a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:16:38.665340 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.665323    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-kube-api-access-zbrkw" (OuterVolumeSpecName: "kube-api-access-zbrkw") pod "877a6520-a5ba-45f6-932e-6b47d170a6a0" (UID: "877a6520-a5ba-45f6-932e-6b47d170a6a0"). InnerVolumeSpecName "kube-api-access-zbrkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:16:38.665340 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.665329    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "877a6520-a5ba-45f6-932e-6b47d170a6a0" (UID: "877a6520-a5ba-45f6-932e-6b47d170a6a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:16:38.763870 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.763840    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:16:38.763870 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.763865    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbrkw\" (UniqueName: \"kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-kube-api-access-zbrkw\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:16:38.763870 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.763875    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:16:38.764105 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.763883    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:16:38.764105 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:38.763890    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/877a6520-a5ba-45f6-932e-6b47d170a6a0-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:16:39.055137 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.055100    2558 generic.go:356] "Generic (PLEG): container finished" podID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerID="1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc" exitCode=0
Jul 17 06:16:39.055327 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.055148    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"
Jul 17 06:16:39.055327 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.055179    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w" event={"ID":"877a6520-a5ba-45f6-932e-6b47d170a6a0","Type":"ContainerDied","Data":"1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc"}
Jul 17 06:16:39.055327 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.055237    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w" event={"ID":"877a6520-a5ba-45f6-932e-6b47d170a6a0","Type":"ContainerDied","Data":"cecc551ff539b3dd985aac7a86b44213e53dfa22b552df22509b810fafb06ea0"}
Jul 17 06:16:39.055327 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.055261    2558 scope.go:117] "RemoveContainer" containerID="1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc"
Jul 17 06:16:39.065597 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.065581    2558 scope.go:117] "RemoveContainer" containerID="621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2"
Jul 17 06:16:39.074796 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.074779    2558 scope.go:117] "RemoveContainer" containerID="1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc"
Jul 17 06:16:39.075068 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:16:39.075043    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc\": container with ID starting with 1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc not found: ID does not exist" containerID="1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc"
Jul 17 06:16:39.075171 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.075077    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc"} err="failed to get container status \"1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc\": rpc error: code = NotFound desc = could not find container \"1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc\": container with ID starting with 1acea1af2ca4b4e8a606987eeac835995feb880fe79dcbefcfe0e8ec4ffb3edc not found: ID does not exist"
Jul 17 06:16:39.075171 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.075095    2558 scope.go:117] "RemoveContainer" containerID="621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2"
Jul 17 06:16:39.075327 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:16:39.075307    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2\": container with ID starting with 621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2 not found: ID does not exist" containerID="621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2"
Jul 17 06:16:39.075371 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.075335    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2"} err="failed to get container status \"621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2\": rpc error: code = NotFound desc = could not find container \"621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2\": container with ID starting with 621428260db1a7d03fd2c6bfc9642375c4280fa58835341d25ba31e9c2c4fbe2 not found: ID does not exist"
Jul 17 06:16:39.079698 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.079677    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"]
Jul 17 06:16:39.083148 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:39.083123    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-a9646996-gw-openshift-default-66886d6fbc-wjm5w"]
Jul 17 06:16:40.079876 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:40.079840    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="877a6520-a5ba-45f6-932e-6b47d170a6a0" path="/var/lib/kubelet/pods/877a6520-a5ba-45f6-932e-6b47d170a6a0/volumes"
Jul 17 06:16:45.229335 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.229300    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"]
Jul 17 06:16:45.230049 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230023    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerName="istio-proxy"
Jul 17 06:16:45.230049 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230050    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerName="istio-proxy"
Jul 17 06:16:45.230278 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230070    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerName="revoke-keys"
Jul 17 06:16:45.230278 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230079    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerName="revoke-keys"
Jul 17 06:16:45.230278 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230107    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerName="revoke-keys"
Jul 17 06:16:45.230278 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230116    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b4c7b87-01e1-42b4-873f-a822639eea19" containerName="revoke-keys"
Jul 17 06:16:45.230278 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230128    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerName="istio-proxy"
Jul 17 06:16:45.230278 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230136    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerName="istio-proxy"
Jul 17 06:16:45.230278 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230157    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerName="revoke-keys"
Jul 17 06:16:45.230278 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230165    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerName="revoke-keys"
Jul 17 06:16:45.230278 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230279    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerName="revoke-keys"
Jul 17 06:16:45.230752 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230292    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerName="istio-proxy"
Jul 17 06:16:45.230752 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230301    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerName="revoke-keys"
Jul 17 06:16:45.230752 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230312    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="877a6520-a5ba-45f6-932e-6b47d170a6a0" containerName="istio-proxy"
Jul 17 06:16:45.230752 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.230320    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerName="revoke-keys"
Jul 17 06:16:45.235394 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.235377    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.237983 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.237963    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-572ab047-openshift-default-dockercfg-5qkht\""
Jul 17 06:16:45.246797 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.246772    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"]
Jul 17 06:16:45.321041 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.321009    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-envoy\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.321185 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.321063    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-token\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.321185 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.321129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/892ae5cf-3368-4feb-84a9-2b81f45330cb-istiod-ca-cert\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.321185 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.321175    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-podinfo\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.321328 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.321227    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-certs\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.321328 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.321279    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-credential-socket\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.321399 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.321334    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-data\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.321399 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.321368    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jhms\" (UniqueName: \"kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-kube-api-access-6jhms\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.321486 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.321425    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-socket\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422264 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422231    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-socket\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422431 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422284    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-envoy\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422431 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422315    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-token\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422431 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422334    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/892ae5cf-3368-4feb-84a9-2b81f45330cb-istiod-ca-cert\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422431 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422356    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-podinfo\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422431 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422413    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-certs\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422764 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422452    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-credential-socket\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422764 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-data\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422764 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422559    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jhms\" (UniqueName: \"kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-kube-api-access-6jhms\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422764 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422670    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-socket\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422963 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422876    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-data\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.422963 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.422944    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-certs\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.423516 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.423462    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-credential-socket\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.423849 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.423819    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/892ae5cf-3368-4feb-84a9-2b81f45330cb-istiod-ca-cert\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.425172 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.425147    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-envoy\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.425441 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.425419    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-podinfo\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.431178 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.431151    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-token\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.431581 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.431561    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jhms\" (UniqueName: \"kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-kube-api-access-6jhms\") pod \"e2e-ait-572ab047-openshift-default-594468b699-r84cs\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") " pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.548767 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.548688    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:45.693581 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.693554    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"]
Jul 17 06:16:45.695409 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:16:45.695382    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod892ae5cf_3368_4feb_84a9_2b81f45330cb.slice/crio-20daa7883452aaa1210bc397934ba489b10a456bdcbf28bfe11062a84d02117a WatchSource:0}: Error finding container 20daa7883452aaa1210bc397934ba489b10a456bdcbf28bfe11062a84d02117a: Status 404 returned error can't find the container with id 20daa7883452aaa1210bc397934ba489b10a456bdcbf28bfe11062a84d02117a
Jul 17 06:16:45.697585 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.697549    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:16:45.697655 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.697633    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:16:45.697694 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:45.697683    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:16:46.096926 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:46.096892    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" event={"ID":"892ae5cf-3368-4feb-84a9-2b81f45330cb","Type":"ContainerStarted","Data":"bf97542104fb9b7534afdf810652b47d15d4f19a0e7b496f71f08bf8c471f18a"}
Jul 17 06:16:46.097108 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:46.096937    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" event={"ID":"892ae5cf-3368-4feb-84a9-2b81f45330cb","Type":"ContainerStarted","Data":"20daa7883452aaa1210bc397934ba489b10a456bdcbf28bfe11062a84d02117a"}
Jul 17 06:16:46.131863 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:46.131812    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" podStartSLOduration=1.13179584 podStartE2EDuration="1.13179584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:16:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:16:46.129394287 +0000 UTC m=+2128.571332865" watchObservedRunningTime="2026-07-17 06:16:46.13179584 +0000 UTC m=+2128.573734415"
Jul 17 06:16:46.549113 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:46.549031    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:46.554012 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:46.553988    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:47.101978 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:47.101948    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:47.102825 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:47.102808    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:58.174274 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:58.174239    2558 generic.go:356] "Generic (PLEG): container finished" podID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerID="bf97542104fb9b7534afdf810652b47d15d4f19a0e7b496f71f08bf8c471f18a" exitCode=0
Jul 17 06:16:58.174704 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:58.174308    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" event={"ID":"892ae5cf-3368-4feb-84a9-2b81f45330cb","Type":"ContainerDied","Data":"bf97542104fb9b7534afdf810652b47d15d4f19a0e7b496f71f08bf8c471f18a"}
Jul 17 06:16:58.174704 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:58.174698    2558 scope.go:117] "RemoveContainer" containerID="bf97542104fb9b7534afdf810652b47d15d4f19a0e7b496f71f08bf8c471f18a"
Jul 17 06:16:58.175665 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:58.175642    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:16:58.175725 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:58.175708    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:16:58.175760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:58.175740    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:16:59.181423 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:59.181388    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" event={"ID":"892ae5cf-3368-4feb-84a9-2b81f45330cb","Type":"ContainerStarted","Data":"c3eee1c56c89cf3ede38f2a412d5e8642c80d640bfaf66692c0f443945436d5c"}
Jul 17 06:16:59.549365 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:59.549262    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:16:59.550730 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:59.550692    2558 patch_prober.go:28] interesting pod/e2e-ait-572ab047-openshift-default-594468b699-r84cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 17 06:16:59.550866 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:16:59.550758    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 17 06:17:00.549725 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:17:00.549687    2558 patch_prober.go:28] interesting pod/e2e-ait-572ab047-openshift-default-594468b699-r84cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 17 06:17:00.550224 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:17:00.549802    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 17 06:17:01.550118 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:17:01.550075    2558 patch_prober.go:28] interesting pod/e2e-ait-572ab047-openshift-default-594468b699-r84cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 17 06:17:01.550579 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:17:01.550149    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 17 06:17:02.102554 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:17:02.102528    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:17:02.549358 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:17:02.549265    2558 patch_prober.go:28] interesting pod/e2e-ait-572ab047-openshift-default-594468b699-r84cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 17 06:17:02.549358 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:17:02.549331    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 17 06:17:03.553868 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:17:03.553839    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:17:03.554827 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:17:03.554807    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:18:41.824426 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:41.824344    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"]
Jul 17 06:18:41.825011 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:41.824991    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerName="revoke-keys"
Jul 17 06:18:41.825098 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:41.825015    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a9a71cd-7afc-4111-af26-f0342d9e8d51" containerName="revoke-keys"
Jul 17 06:18:41.827666 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:41.827645    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"
Jul 17 06:18:41.830979 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:41.830955    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:18:41.841347 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:41.841327    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"]
Jul 17 06:18:41.860249 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:41.860220    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9ef322-bd3b-42b0-878c-8d6b803813b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-4wbr8\" (UID: \"7c9ef322-bd3b-42b0-878c-8d6b803813b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"
Jul 17 06:18:41.960816 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:41.960785    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9ef322-bd3b-42b0-878c-8d6b803813b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-4wbr8\" (UID: \"7c9ef322-bd3b-42b0-878c-8d6b803813b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"
Jul 17 06:18:41.961382 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:41.961363    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9ef322-bd3b-42b0-878c-8d6b803813b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-4wbr8\" (UID: \"7c9ef322-bd3b-42b0-878c-8d6b803813b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"
Jul 17 06:18:42.140583 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:42.140547    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"
Jul 17 06:18:42.492962 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:42.492940    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"]
Jul 17 06:18:42.495263 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:18:42.495234    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9ef322_bd3b_42b0_878c_8d6b803813b2.slice/crio-4a15d0f55f2a796b91e64e75e26b6ba95b668fed0670bc1d4e8602c6e5b6c483 WatchSource:0}: Error finding container 4a15d0f55f2a796b91e64e75e26b6ba95b668fed0670bc1d4e8602c6e5b6c483: Status 404 returned error can't find the container with id 4a15d0f55f2a796b91e64e75e26b6ba95b668fed0670bc1d4e8602c6e5b6c483
Jul 17 06:18:42.734632 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:42.734604    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8" event={"ID":"7c9ef322-bd3b-42b0-878c-8d6b803813b2","Type":"ContainerStarted","Data":"4a15d0f55f2a796b91e64e75e26b6ba95b668fed0670bc1d4e8602c6e5b6c483"}
Jul 17 06:18:43.741739 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:43.741696    2558 generic.go:356] "Generic (PLEG): container finished" podID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerID="0aaff990d8689a0dc21ec4947152ea4b1b24b48bcc7c67f511565034c09bfe4b" exitCode=7
Jul 17 06:18:43.742134 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:43.741794    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8" event={"ID":"7c9ef322-bd3b-42b0-878c-8d6b803813b2","Type":"ContainerDied","Data":"0aaff990d8689a0dc21ec4947152ea4b1b24b48bcc7c67f511565034c09bfe4b"}
Jul 17 06:18:43.742134 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:43.741970    2558 scope.go:117] "RemoveContainer" containerID="0aaff990d8689a0dc21ec4947152ea4b1b24b48bcc7c67f511565034c09bfe4b"
Jul 17 06:18:44.749990 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:44.749952    2558 generic.go:356] "Generic (PLEG): container finished" podID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerID="198549b714b2529a401b247938c8cd64028b8dd67efceef2421561c85f2b04f7" exitCode=7
Jul 17 06:18:44.750508 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:44.750037    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8" event={"ID":"7c9ef322-bd3b-42b0-878c-8d6b803813b2","Type":"ContainerDied","Data":"198549b714b2529a401b247938c8cd64028b8dd67efceef2421561c85f2b04f7"}
Jul 17 06:18:44.750508 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:44.750128    2558 scope.go:117] "RemoveContainer" containerID="0aaff990d8689a0dc21ec4947152ea4b1b24b48bcc7c67f511565034c09bfe4b"
Jul 17 06:18:44.750508 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:44.750383    2558 scope.go:117] "RemoveContainer" containerID="198549b714b2529a401b247938c8cd64028b8dd67efceef2421561c85f2b04f7"
Jul 17 06:18:44.750692 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:18:44.750641    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-4wbr8_odh-ai-gateway-infra(7c9ef322-bd3b-42b0-878c-8d6b803813b2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2"
Jul 17 06:18:45.487502 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:45.487454    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"]
Jul 17 06:18:45.491068 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:45.491052    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"
Jul 17 06:18:45.501711 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:45.501681    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"]
Jul 17 06:18:45.602743 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:45.602718    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f59e78a2-3792-4552-86e8-5e2205c39353-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8\" (UID: \"f59e78a2-3792-4552-86e8-5e2205c39353\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"
Jul 17 06:18:45.703908 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:45.703880    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f59e78a2-3792-4552-86e8-5e2205c39353-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8\" (UID: \"f59e78a2-3792-4552-86e8-5e2205c39353\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"
Jul 17 06:18:45.704596 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:45.704578    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f59e78a2-3792-4552-86e8-5e2205c39353-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8\" (UID: \"f59e78a2-3792-4552-86e8-5e2205c39353\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"
Jul 17 06:18:45.757865 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:45.757811    2558 scope.go:117] "RemoveContainer" containerID="198549b714b2529a401b247938c8cd64028b8dd67efceef2421561c85f2b04f7"
Jul 17 06:18:45.758158 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:18:45.757994    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-4wbr8_odh-ai-gateway-infra(7c9ef322-bd3b-42b0-878c-8d6b803813b2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2"
Jul 17 06:18:45.802639 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:45.802618    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"
Jul 17 06:18:46.152809 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:46.152775    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"]
Jul 17 06:18:46.153612 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:18:46.153577    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf59e78a2_3792_4552_86e8_5e2205c39353.slice/crio-5f39f0f38a819d5180ab50444e688df6603628e0462c4dd47d8904542366b53e WatchSource:0}: Error finding container 5f39f0f38a819d5180ab50444e688df6603628e0462c4dd47d8904542366b53e: Status 404 returned error can't find the container with id 5f39f0f38a819d5180ab50444e688df6603628e0462c4dd47d8904542366b53e
Jul 17 06:18:46.764679 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:46.764654    2558 generic.go:356] "Generic (PLEG): container finished" podID="f59e78a2-3792-4552-86e8-5e2205c39353" containerID="0e7473459d40af42d0d34a30f51a2122fb944b2d5362bd0fe50dc78500fef89c" exitCode=6
Jul 17 06:18:46.764996 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:46.764690    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8" event={"ID":"f59e78a2-3792-4552-86e8-5e2205c39353","Type":"ContainerDied","Data":"0e7473459d40af42d0d34a30f51a2122fb944b2d5362bd0fe50dc78500fef89c"}
Jul 17 06:18:46.764996 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:46.764753    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8" event={"ID":"f59e78a2-3792-4552-86e8-5e2205c39353","Type":"ContainerStarted","Data":"5f39f0f38a819d5180ab50444e688df6603628e0462c4dd47d8904542366b53e"}
Jul 17 06:18:46.764996 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:46.764912    2558 scope.go:117] "RemoveContainer" containerID="0e7473459d40af42d0d34a30f51a2122fb944b2d5362bd0fe50dc78500fef89c"
Jul 17 06:18:47.772300 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:47.772213    2558 generic.go:356] "Generic (PLEG): container finished" podID="f59e78a2-3792-4552-86e8-5e2205c39353" containerID="3b55627ac17643f773bfe3edbf913b143dcda3b8824612f8dd57431ada805744" exitCode=6
Jul 17 06:18:47.772300 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:47.772263    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8" event={"ID":"f59e78a2-3792-4552-86e8-5e2205c39353","Type":"ContainerDied","Data":"3b55627ac17643f773bfe3edbf913b143dcda3b8824612f8dd57431ada805744"}
Jul 17 06:18:47.772793 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:47.772314    2558 scope.go:117] "RemoveContainer" containerID="0e7473459d40af42d0d34a30f51a2122fb944b2d5362bd0fe50dc78500fef89c"
Jul 17 06:18:47.772793 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:47.772558    2558 scope.go:117] "RemoveContainer" containerID="3b55627ac17643f773bfe3edbf913b143dcda3b8824612f8dd57431ada805744"
Jul 17 06:18:47.772876 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:18:47.772793    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8_odh-ai-gateway-infra(f59e78a2-3792-4552-86e8-5e2205c39353)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8" podUID="f59e78a2-3792-4552-86e8-5e2205c39353"
Jul 17 06:18:48.779679 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:18:48.779651    2558 scope.go:117] "RemoveContainer" containerID="3b55627ac17643f773bfe3edbf913b143dcda3b8824612f8dd57431ada805744"
Jul 17 06:18:48.780067 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:18:48.779837    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8_odh-ai-gateway-infra(f59e78a2-3792-4552-86e8-5e2205c39353)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8" podUID="f59e78a2-3792-4552-86e8-5e2205c39353"
Jul 17 06:19:01.073179 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:01.073147    2558 scope.go:117] "RemoveContainer" containerID="198549b714b2529a401b247938c8cd64028b8dd67efceef2421561c85f2b04f7"
Jul 17 06:19:01.852141 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:01.852107    2558 generic.go:356] "Generic (PLEG): container finished" podID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerID="9c156e8c0655c1afd08a2dfaa8b5b9f62f5301aaa5c4c30822b7d1dd64db17dd" exitCode=7
Jul 17 06:19:01.852319 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:01.852183    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8" event={"ID":"7c9ef322-bd3b-42b0-878c-8d6b803813b2","Type":"ContainerDied","Data":"9c156e8c0655c1afd08a2dfaa8b5b9f62f5301aaa5c4c30822b7d1dd64db17dd"}
Jul 17 06:19:01.852319 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:01.852248    2558 scope.go:117] "RemoveContainer" containerID="198549b714b2529a401b247938c8cd64028b8dd67efceef2421561c85f2b04f7"
Jul 17 06:19:01.852586 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:01.852567    2558 scope.go:117] "RemoveContainer" containerID="9c156e8c0655c1afd08a2dfaa8b5b9f62f5301aaa5c4c30822b7d1dd64db17dd"
Jul 17 06:19:01.852845 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:19:01.852826    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-4wbr8_odh-ai-gateway-infra(7c9ef322-bd3b-42b0-878c-8d6b803813b2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2"
Jul 17 06:19:02.892204 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:02.892169    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"]
Jul 17 06:19:03.051351 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.051328    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"
Jul 17 06:19:03.072438 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.072411    2558 scope.go:117] "RemoveContainer" containerID="3b55627ac17643f773bfe3edbf913b143dcda3b8824612f8dd57431ada805744"
Jul 17 06:19:03.162539 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.162445    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9ef322-bd3b-42b0-878c-8d6b803813b2-maas-api-service-ca\") pod \"7c9ef322-bd3b-42b0-878c-8d6b803813b2\" (UID: \"7c9ef322-bd3b-42b0-878c-8d6b803813b2\") "
Jul 17 06:19:03.162811 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.162773    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9ef322-bd3b-42b0-878c-8d6b803813b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7c9ef322-bd3b-42b0-878c-8d6b803813b2" (UID: "7c9ef322-bd3b-42b0-878c-8d6b803813b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:19:03.163271 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.163239    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9ef322-bd3b-42b0-878c-8d6b803813b2-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:19:03.866213 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.866183    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"
Jul 17 06:19:03.866213 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.866204    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8" event={"ID":"7c9ef322-bd3b-42b0-878c-8d6b803813b2","Type":"ContainerDied","Data":"4a15d0f55f2a796b91e64e75e26b6ba95b668fed0670bc1d4e8602c6e5b6c483"}
Jul 17 06:19:03.866461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.866267    2558 scope.go:117] "RemoveContainer" containerID="9c156e8c0655c1afd08a2dfaa8b5b9f62f5301aaa5c4c30822b7d1dd64db17dd"
Jul 17 06:19:03.868440 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.868416    2558 generic.go:356] "Generic (PLEG): container finished" podID="f59e78a2-3792-4552-86e8-5e2205c39353" containerID="d1ad86de10054a7eee9ff45cf6701b218adfc6d51006779efbed18ed1b377d3f" exitCode=6
Jul 17 06:19:03.868604 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.868460    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8" event={"ID":"f59e78a2-3792-4552-86e8-5e2205c39353","Type":"ContainerDied","Data":"d1ad86de10054a7eee9ff45cf6701b218adfc6d51006779efbed18ed1b377d3f"}
Jul 17 06:19:03.868791 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.868774    2558 scope.go:117] "RemoveContainer" containerID="d1ad86de10054a7eee9ff45cf6701b218adfc6d51006779efbed18ed1b377d3f"
Jul 17 06:19:03.869005 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:19:03.868984    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8_odh-ai-gateway-infra(f59e78a2-3792-4552-86e8-5e2205c39353)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8" podUID="f59e78a2-3792-4552-86e8-5e2205c39353"
Jul 17 06:19:03.877463 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.877445    2558 scope.go:117] "RemoveContainer" containerID="3b55627ac17643f773bfe3edbf913b143dcda3b8824612f8dd57431ada805744"
Jul 17 06:19:03.954073 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.954035    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"]
Jul 17 06:19:03.978812 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:03.978783    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-4wbr8"]
Jul 17 06:19:04.079820 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:04.079783    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" path="/var/lib/kubelet/pods/7c9ef322-bd3b-42b0-878c-8d6b803813b2/volumes"
Jul 17 06:19:04.930467 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:04.930435    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"]
Jul 17 06:19:05.065145 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:05.065115    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"
Jul 17 06:19:05.182080 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:05.181978    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f59e78a2-3792-4552-86e8-5e2205c39353-maas-api-service-ca\") pod \"f59e78a2-3792-4552-86e8-5e2205c39353\" (UID: \"f59e78a2-3792-4552-86e8-5e2205c39353\") "
Jul 17 06:19:05.182340 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:05.182318    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f59e78a2-3792-4552-86e8-5e2205c39353-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f59e78a2-3792-4552-86e8-5e2205c39353" (UID: "f59e78a2-3792-4552-86e8-5e2205c39353"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:19:05.283275 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:05.283234    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f59e78a2-3792-4552-86e8-5e2205c39353-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:19:05.883348 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:05.883316    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"
Jul 17 06:19:05.883578 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:05.883316    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8" event={"ID":"f59e78a2-3792-4552-86e8-5e2205c39353","Type":"ContainerDied","Data":"5f39f0f38a819d5180ab50444e688df6603628e0462c4dd47d8904542366b53e"}
Jul 17 06:19:05.883578 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:05.883504    2558 scope.go:117] "RemoveContainer" containerID="d1ad86de10054a7eee9ff45cf6701b218adfc6d51006779efbed18ed1b377d3f"
Jul 17 06:19:05.908220 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:05.908191    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"]
Jul 17 06:19:05.910816 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:05.910790    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-r4fw8"]
Jul 17 06:19:06.078684 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:19:06.078650    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f59e78a2-3792-4552-86e8-5e2205c39353" path="/var/lib/kubelet/pods/f59e78a2-3792-4552-86e8-5e2205c39353/volumes"
Jul 17 06:20:45.598041 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598014    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"]
Jul 17 06:20:45.598504 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598442    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerName="revoke-keys"
Jul 17 06:20:45.598504 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598453    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerName="revoke-keys"
Jul 17 06:20:45.598504 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598461    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerName="revoke-keys"
Jul 17 06:20:45.598504 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598466    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerName="revoke-keys"
Jul 17 06:20:45.598504 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598486    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f59e78a2-3792-4552-86e8-5e2205c39353" containerName="revoke-keys"
Jul 17 06:20:45.598504 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598492    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f59e78a2-3792-4552-86e8-5e2205c39353" containerName="revoke-keys"
Jul 17 06:20:45.598504 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598500    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f59e78a2-3792-4552-86e8-5e2205c39353" containerName="revoke-keys"
Jul 17 06:20:45.598504 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598506    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f59e78a2-3792-4552-86e8-5e2205c39353" containerName="revoke-keys"
Jul 17 06:20:45.598776 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598515    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f59e78a2-3792-4552-86e8-5e2205c39353" containerName="revoke-keys"
Jul 17 06:20:45.598776 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598520    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f59e78a2-3792-4552-86e8-5e2205c39353" containerName="revoke-keys"
Jul 17 06:20:45.598776 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598588    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerName="revoke-keys"
Jul 17 06:20:45.598776 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598596    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f59e78a2-3792-4552-86e8-5e2205c39353" containerName="revoke-keys"
Jul 17 06:20:45.598776 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598603    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerName="revoke-keys"
Jul 17 06:20:45.598776 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598610    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f59e78a2-3792-4552-86e8-5e2205c39353" containerName="revoke-keys"
Jul 17 06:20:45.598776 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.598616    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f59e78a2-3792-4552-86e8-5e2205c39353" containerName="revoke-keys"
Jul 17 06:20:45.600774 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.600758    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"
Jul 17 06:20:45.604047 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.604026    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:20:45.611822 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.611802    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"]
Jul 17 06:20:45.694292 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.694269    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fa43-4f08-476d-9b7a-0ca66a515455-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-vrm79\" (UID: \"7ca7fa43-4f08-476d-9b7a-0ca66a515455\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"
Jul 17 06:20:45.795641 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.795614    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fa43-4f08-476d-9b7a-0ca66a515455-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-vrm79\" (UID: \"7ca7fa43-4f08-476d-9b7a-0ca66a515455\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"
Jul 17 06:20:45.796217 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.796200    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fa43-4f08-476d-9b7a-0ca66a515455-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-vrm79\" (UID: \"7ca7fa43-4f08-476d-9b7a-0ca66a515455\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"
Jul 17 06:20:45.911887 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:45.911797    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"
Jul 17 06:20:46.076619 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:20:46.076588    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ca7fa43_4f08_476d_9b7a_0ca66a515455.slice/crio-8c490e86888d73c769a9cffbc783f5310215b8f619987ba2b4f411dee2caa295 WatchSource:0}: Error finding container 8c490e86888d73c769a9cffbc783f5310215b8f619987ba2b4f411dee2caa295: Status 404 returned error can't find the container with id 8c490e86888d73c769a9cffbc783f5310215b8f619987ba2b4f411dee2caa295
Jul 17 06:20:46.079907 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:46.079884    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"]
Jul 17 06:20:46.422034 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:46.421995    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79" event={"ID":"7ca7fa43-4f08-476d-9b7a-0ca66a515455","Type":"ContainerStarted","Data":"8e6ce905489379de3c3ff8e79bd0544e167bf6117a7af45ed90ebe90705b97ea"}
Jul 17 06:20:46.422221 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:46.422052    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79" event={"ID":"7ca7fa43-4f08-476d-9b7a-0ca66a515455","Type":"ContainerStarted","Data":"8c490e86888d73c769a9cffbc783f5310215b8f619987ba2b4f411dee2caa295"}
Jul 17 06:20:46.449628 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:46.449565    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79" podStartSLOduration=1.449549122 podStartE2EDuration="1.449549122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:20:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:20:46.448904234 +0000 UTC m=+2368.890842964" watchObservedRunningTime="2026-07-17 06:20:46.449549122 +0000 UTC m=+2368.891487761"
Jul 17 06:20:47.428488 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:47.428444    2558 generic.go:356] "Generic (PLEG): container finished" podID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerID="8e6ce905489379de3c3ff8e79bd0544e167bf6117a7af45ed90ebe90705b97ea" exitCode=6
Jul 17 06:20:47.428990 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:47.428544    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79" event={"ID":"7ca7fa43-4f08-476d-9b7a-0ca66a515455","Type":"ContainerDied","Data":"8e6ce905489379de3c3ff8e79bd0544e167bf6117a7af45ed90ebe90705b97ea"}
Jul 17 06:20:47.428990 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:47.428821    2558 scope.go:117] "RemoveContainer" containerID="8e6ce905489379de3c3ff8e79bd0544e167bf6117a7af45ed90ebe90705b97ea"
Jul 17 06:20:48.435735 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:48.435647    2558 generic.go:356] "Generic (PLEG): container finished" podID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerID="beb51200aad61f22042a36cd885d6b591962470ff486c453c0754dc0dbf60150" exitCode=6
Jul 17 06:20:48.436198 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:48.435736    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79" event={"ID":"7ca7fa43-4f08-476d-9b7a-0ca66a515455","Type":"ContainerDied","Data":"beb51200aad61f22042a36cd885d6b591962470ff486c453c0754dc0dbf60150"}
Jul 17 06:20:48.436198 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:48.435801    2558 scope.go:117] "RemoveContainer" containerID="8e6ce905489379de3c3ff8e79bd0544e167bf6117a7af45ed90ebe90705b97ea"
Jul 17 06:20:48.436198 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:48.436043    2558 scope.go:117] "RemoveContainer" containerID="beb51200aad61f22042a36cd885d6b591962470ff486c453c0754dc0dbf60150"
Jul 17 06:20:48.436349 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:20:48.436324    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-vrm79_odh-ai-gateway-infra(7ca7fa43-4f08-476d-9b7a-0ca66a515455)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455"
Jul 17 06:20:49.443582 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:20:49.443552    2558 scope.go:117] "RemoveContainer" containerID="beb51200aad61f22042a36cd885d6b591962470ff486c453c0754dc0dbf60150"
Jul 17 06:20:49.443981 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:20:49.443774    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-vrm79_odh-ai-gateway-infra(7ca7fa43-4f08-476d-9b7a-0ca66a515455)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455"
Jul 17 06:21:03.072941 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:03.072907    2558 scope.go:117] "RemoveContainer" containerID="beb51200aad61f22042a36cd885d6b591962470ff486c453c0754dc0dbf60150"
Jul 17 06:21:03.524213 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:03.524181    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79" event={"ID":"7ca7fa43-4f08-476d-9b7a-0ca66a515455","Type":"ContainerStarted","Data":"44a08e12b20354e5193a131b73c4e96dd78554477c2a3d601bc4ec4aea674c8b"}
Jul 17 06:21:04.531680 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:04.531642    2558 generic.go:356] "Generic (PLEG): container finished" podID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerID="44a08e12b20354e5193a131b73c4e96dd78554477c2a3d601bc4ec4aea674c8b" exitCode=6
Jul 17 06:21:04.532126 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:04.531724    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79" event={"ID":"7ca7fa43-4f08-476d-9b7a-0ca66a515455","Type":"ContainerDied","Data":"44a08e12b20354e5193a131b73c4e96dd78554477c2a3d601bc4ec4aea674c8b"}
Jul 17 06:21:04.532126 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:04.531804    2558 scope.go:117] "RemoveContainer" containerID="beb51200aad61f22042a36cd885d6b591962470ff486c453c0754dc0dbf60150"
Jul 17 06:21:04.532249 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:04.532158    2558 scope.go:117] "RemoveContainer" containerID="44a08e12b20354e5193a131b73c4e96dd78554477c2a3d601bc4ec4aea674c8b"
Jul 17 06:21:04.532433 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:21:04.532410    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-vrm79_odh-ai-gateway-infra(7ca7fa43-4f08-476d-9b7a-0ca66a515455)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455"
Jul 17 06:21:04.572158 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:04.572119    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"]
Jul 17 06:21:05.685405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:05.685377    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"
Jul 17 06:21:05.776030 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:05.775991    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fa43-4f08-476d-9b7a-0ca66a515455-maas-api-service-ca\") pod \"7ca7fa43-4f08-476d-9b7a-0ca66a515455\" (UID: \"7ca7fa43-4f08-476d-9b7a-0ca66a515455\") "
Jul 17 06:21:05.776397 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:05.776369    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ca7fa43-4f08-476d-9b7a-0ca66a515455-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ca7fa43-4f08-476d-9b7a-0ca66a515455" (UID: "7ca7fa43-4f08-476d-9b7a-0ca66a515455"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:21:05.877795 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:05.877759    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fa43-4f08-476d-9b7a-0ca66a515455-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:21:06.545973 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:06.545941    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79" event={"ID":"7ca7fa43-4f08-476d-9b7a-0ca66a515455","Type":"ContainerDied","Data":"8c490e86888d73c769a9cffbc783f5310215b8f619987ba2b4f411dee2caa295"}
Jul 17 06:21:06.545973 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:06.545971    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"
Jul 17 06:21:06.546211 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:06.546004    2558 scope.go:117] "RemoveContainer" containerID="44a08e12b20354e5193a131b73c4e96dd78554477c2a3d601bc4ec4aea674c8b"
Jul 17 06:21:06.567605 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:06.567575    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"]
Jul 17 06:21:06.571545 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:06.571519    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vrm79"]
Jul 17 06:21:08.079132 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:08.079086    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" path="/var/lib/kubelet/pods/7ca7fa43-4f08-476d-9b7a-0ca66a515455/volumes"
Jul 17 06:21:18.166211 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:18.166181    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:21:18.167834 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:18.167807    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:21:18.172488 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:18.172446    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:21:18.173823 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:18.173800    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:21:45.673415 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.673340    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"]
Jul 17 06:21:45.673964 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.673945    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerName="revoke-keys"
Jul 17 06:21:45.674047 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.673967    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerName="revoke-keys"
Jul 17 06:21:45.674047 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.673980    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerName="revoke-keys"
Jul 17 06:21:45.674047 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.673988    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerName="revoke-keys"
Jul 17 06:21:45.674047 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.674034    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerName="revoke-keys"
Jul 17 06:21:45.674047 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.674043    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerName="revoke-keys"
Jul 17 06:21:45.674291 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.674142    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerName="revoke-keys"
Jul 17 06:21:45.674291 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.674154    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerName="revoke-keys"
Jul 17 06:21:45.674291 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.674164    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c9ef322-bd3b-42b0-878c-8d6b803813b2" containerName="revoke-keys"
Jul 17 06:21:45.677399 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.677379    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"
Jul 17 06:21:45.681159 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.681141    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:21:45.690465 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.690440    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"]
Jul 17 06:21:45.746355 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.746326    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5070c26-0351-4080-9d41-fb60c387b2b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz\" (UID: \"a5070c26-0351-4080-9d41-fb60c387b2b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"
Jul 17 06:21:45.847420 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.847394    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5070c26-0351-4080-9d41-fb60c387b2b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz\" (UID: \"a5070c26-0351-4080-9d41-fb60c387b2b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"
Jul 17 06:21:45.848066 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.848046    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5070c26-0351-4080-9d41-fb60c387b2b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz\" (UID: \"a5070c26-0351-4080-9d41-fb60c387b2b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"
Jul 17 06:21:45.989639 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:45.989584    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"
Jul 17 06:21:46.132533 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:46.132506    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"]
Jul 17 06:21:46.134599 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:21:46.134567    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5070c26_0351_4080_9d41_fb60c387b2b4.slice/crio-3e74c178756cc8db7976d7b276c0ef31e399677b1f86b106557b10728738e890 WatchSource:0}: Error finding container 3e74c178756cc8db7976d7b276c0ef31e399677b1f86b106557b10728738e890: Status 404 returned error can't find the container with id 3e74c178756cc8db7976d7b276c0ef31e399677b1f86b106557b10728738e890
Jul 17 06:21:46.763908 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:46.763880    2558 generic.go:356] "Generic (PLEG): container finished" podID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerID="bf8ca2e63282c6a9fc283bf73118668d0daf2c45785510b74b2d5e5fa785fd9c" exitCode=6
Jul 17 06:21:46.764226 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:46.763967    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz" event={"ID":"a5070c26-0351-4080-9d41-fb60c387b2b4","Type":"ContainerDied","Data":"bf8ca2e63282c6a9fc283bf73118668d0daf2c45785510b74b2d5e5fa785fd9c"}
Jul 17 06:21:46.764226 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:46.764029    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz" event={"ID":"a5070c26-0351-4080-9d41-fb60c387b2b4","Type":"ContainerStarted","Data":"3e74c178756cc8db7976d7b276c0ef31e399677b1f86b106557b10728738e890"}
Jul 17 06:21:46.764226 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:46.764127    2558 scope.go:117] "RemoveContainer" containerID="bf8ca2e63282c6a9fc283bf73118668d0daf2c45785510b74b2d5e5fa785fd9c"
Jul 17 06:21:47.771305 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:47.771219    2558 generic.go:356] "Generic (PLEG): container finished" podID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerID="fd249be1a57cf3a9547bf2556148bf55d8c062f90729ed04646b92a4108a7550" exitCode=6
Jul 17 06:21:47.771682 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:47.771299    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz" event={"ID":"a5070c26-0351-4080-9d41-fb60c387b2b4","Type":"ContainerDied","Data":"fd249be1a57cf3a9547bf2556148bf55d8c062f90729ed04646b92a4108a7550"}
Jul 17 06:21:47.771682 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:47.771355    2558 scope.go:117] "RemoveContainer" containerID="bf8ca2e63282c6a9fc283bf73118668d0daf2c45785510b74b2d5e5fa785fd9c"
Jul 17 06:21:47.771682 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:47.771555    2558 scope.go:117] "RemoveContainer" containerID="fd249be1a57cf3a9547bf2556148bf55d8c062f90729ed04646b92a4108a7550"
Jul 17 06:21:47.771835 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:21:47.771761    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz_odh-ai-gateway-infra(a5070c26-0351-4080-9d41-fb60c387b2b4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4"
Jul 17 06:21:48.778383 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:21:48.778354    2558 scope.go:117] "RemoveContainer" containerID="fd249be1a57cf3a9547bf2556148bf55d8c062f90729ed04646b92a4108a7550"
Jul 17 06:21:48.778792 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:21:48.778567    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz_odh-ai-gateway-infra(a5070c26-0351-4080-9d41-fb60c387b2b4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4"
Jul 17 06:22:04.072214 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:04.072177    2558 scope.go:117] "RemoveContainer" containerID="fd249be1a57cf3a9547bf2556148bf55d8c062f90729ed04646b92a4108a7550"
Jul 17 06:22:04.866713 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:04.866674    2558 generic.go:356] "Generic (PLEG): container finished" podID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerID="7dab492d9e905bf0837fd4c6f8b630cae211b47189dc1a9e1630b7d463742612" exitCode=6
Jul 17 06:22:04.866884 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:04.866750    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz" event={"ID":"a5070c26-0351-4080-9d41-fb60c387b2b4","Type":"ContainerDied","Data":"7dab492d9e905bf0837fd4c6f8b630cae211b47189dc1a9e1630b7d463742612"}
Jul 17 06:22:04.866884 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:04.866816    2558 scope.go:117] "RemoveContainer" containerID="fd249be1a57cf3a9547bf2556148bf55d8c062f90729ed04646b92a4108a7550"
Jul 17 06:22:04.867151 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:04.867125    2558 scope.go:117] "RemoveContainer" containerID="7dab492d9e905bf0837fd4c6f8b630cae211b47189dc1a9e1630b7d463742612"
Jul 17 06:22:04.867362 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:22:04.867341    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz_odh-ai-gateway-infra(a5070c26-0351-4080-9d41-fb60c387b2b4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4"
Jul 17 06:22:05.919636 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:05.919589    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"]
Jul 17 06:22:06.069096 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:06.069064    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"
Jul 17 06:22:06.119654 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:06.119608    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5070c26-0351-4080-9d41-fb60c387b2b4-maas-api-service-ca\") pod \"a5070c26-0351-4080-9d41-fb60c387b2b4\" (UID: \"a5070c26-0351-4080-9d41-fb60c387b2b4\") "
Jul 17 06:22:06.119984 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:06.119958    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5070c26-0351-4080-9d41-fb60c387b2b4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a5070c26-0351-4080-9d41-fb60c387b2b4" (UID: "a5070c26-0351-4080-9d41-fb60c387b2b4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:22:06.221239 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:06.221148    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5070c26-0351-4080-9d41-fb60c387b2b4-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:22:06.880952 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:06.880921    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"
Jul 17 06:22:06.880952 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:06.880949    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz" event={"ID":"a5070c26-0351-4080-9d41-fb60c387b2b4","Type":"ContainerDied","Data":"3e74c178756cc8db7976d7b276c0ef31e399677b1f86b106557b10728738e890"}
Jul 17 06:22:06.881203 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:06.881004    2558 scope.go:117] "RemoveContainer" containerID="7dab492d9e905bf0837fd4c6f8b630cae211b47189dc1a9e1630b7d463742612"
Jul 17 06:22:06.902932 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:06.902902    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"]
Jul 17 06:22:06.909424 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:06.909402    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-vc9qz"]
Jul 17 06:22:08.079562 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:08.079522    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4" path="/var/lib/kubelet/pods/a5070c26-0351-4080-9d41-fb60c387b2b4/volumes"
Jul 17 06:22:32.037404 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.037367    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"]
Jul 17 06:22:32.037879 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.037787    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerName="revoke-keys"
Jul 17 06:22:32.037879 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.037798    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerName="revoke-keys"
Jul 17 06:22:32.037879 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.037811    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerName="revoke-keys"
Jul 17 06:22:32.037879 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.037817    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerName="revoke-keys"
Jul 17 06:22:32.037879 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.037836    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerName="revoke-keys"
Jul 17 06:22:32.037879 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.037843    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerName="revoke-keys"
Jul 17 06:22:32.038072 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.037897    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerName="revoke-keys"
Jul 17 06:22:32.038072 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.037906    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ca7fa43-4f08-476d-9b7a-0ca66a515455" containerName="revoke-keys"
Jul 17 06:22:32.038072 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.037912    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerName="revoke-keys"
Jul 17 06:22:32.038072 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.037917    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerName="revoke-keys"
Jul 17 06:22:32.041005 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.040987    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"
Jul 17 06:22:32.043751 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.043730    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:22:32.050579 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.050557    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"]
Jul 17 06:22:32.053113 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.053082    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-dpsqq\" (UID: \"e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"
Jul 17 06:22:32.154573 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.154540    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-dpsqq\" (UID: \"e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"
Jul 17 06:22:32.155158 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.155135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-dpsqq\" (UID: \"e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"
Jul 17 06:22:32.355478 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.355454    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"
Jul 17 06:22:32.491589 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:32.491555    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"]
Jul 17 06:22:32.492144 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:22:32.492116    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode697cdc0_65fd_46f7_9a15_7fbbd5ee9dc4.slice/crio-a5dfd2a1be589541448e18434c41869f44edc96d4fd7951558444ac49f38c86b WatchSource:0}: Error finding container a5dfd2a1be589541448e18434c41869f44edc96d4fd7951558444ac49f38c86b: Status 404 returned error can't find the container with id a5dfd2a1be589541448e18434c41869f44edc96d4fd7951558444ac49f38c86b
Jul 17 06:22:33.022899 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:33.022819    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq" event={"ID":"e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4","Type":"ContainerStarted","Data":"93851c679600c4aa76bbf41627e3213a1984e47a991af9309b0f3db67b0076c7"}
Jul 17 06:22:33.022899 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:33.022875    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq" event={"ID":"e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4","Type":"ContainerStarted","Data":"a5dfd2a1be589541448e18434c41869f44edc96d4fd7951558444ac49f38c86b"}
Jul 17 06:22:33.039682 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:33.039639    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq" podStartSLOduration=1.039624572 podStartE2EDuration="1.039624572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:22:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:22:33.038165578 +0000 UTC m=+2475.480104164" watchObservedRunningTime="2026-07-17 06:22:33.039624572 +0000 UTC m=+2475.481563146"
Jul 17 06:22:34.030338 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:34.030301    2558 generic.go:356] "Generic (PLEG): container finished" podID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerID="93851c679600c4aa76bbf41627e3213a1984e47a991af9309b0f3db67b0076c7" exitCode=7
Jul 17 06:22:34.030555 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:34.030347    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq" event={"ID":"e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4","Type":"ContainerDied","Data":"93851c679600c4aa76bbf41627e3213a1984e47a991af9309b0f3db67b0076c7"}
Jul 17 06:22:34.030623 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:34.030588    2558 scope.go:117] "RemoveContainer" containerID="93851c679600c4aa76bbf41627e3213a1984e47a991af9309b0f3db67b0076c7"
Jul 17 06:22:35.036965 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:35.036879    2558 generic.go:356] "Generic (PLEG): container finished" podID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerID="002ee044d316938cf37c34207a534c42319e3144265227aaad529525f2e7429c" exitCode=7
Jul 17 06:22:35.037376 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:35.036968    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq" event={"ID":"e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4","Type":"ContainerDied","Data":"002ee044d316938cf37c34207a534c42319e3144265227aaad529525f2e7429c"}
Jul 17 06:22:35.037376 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:35.037032    2558 scope.go:117] "RemoveContainer" containerID="93851c679600c4aa76bbf41627e3213a1984e47a991af9309b0f3db67b0076c7"
Jul 17 06:22:35.037376 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:35.037209    2558 scope.go:117] "RemoveContainer" containerID="002ee044d316938cf37c34207a534c42319e3144265227aaad529525f2e7429c"
Jul 17 06:22:35.037519 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:22:35.037429    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-dpsqq_odh-ai-gateway-infra(e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4"
Jul 17 06:22:36.044203 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:36.044170    2558 scope.go:117] "RemoveContainer" containerID="002ee044d316938cf37c34207a534c42319e3144265227aaad529525f2e7429c"
Jul 17 06:22:36.044595 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:22:36.044365    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-dpsqq_odh-ai-gateway-infra(e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4"
Jul 17 06:22:51.072363 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:51.072332    2558 scope.go:117] "RemoveContainer" containerID="002ee044d316938cf37c34207a534c42319e3144265227aaad529525f2e7429c"
Jul 17 06:22:52.015265 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:52.015186    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"]
Jul 17 06:22:52.015429 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:52.015376    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerName="istio-proxy" containerID="cri-o://c3eee1c56c89cf3ede38f2a412d5e8642c80d640bfaf66692c0f443945436d5c" gracePeriod=30
Jul 17 06:22:52.132027 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:52.131999    2558 generic.go:356] "Generic (PLEG): container finished" podID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerID="722f5e7ecfcc0ef09909696d768100113460246cb381a5460ddbc1af07634c16" exitCode=7
Jul 17 06:22:52.132461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:52.132071    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq" event={"ID":"e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4","Type":"ContainerDied","Data":"722f5e7ecfcc0ef09909696d768100113460246cb381a5460ddbc1af07634c16"}
Jul 17 06:22:52.132461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:52.132129    2558 scope.go:117] "RemoveContainer" containerID="002ee044d316938cf37c34207a534c42319e3144265227aaad529525f2e7429c"
Jul 17 06:22:52.132461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:52.132373    2558 scope.go:117] "RemoveContainer" containerID="722f5e7ecfcc0ef09909696d768100113460246cb381a5460ddbc1af07634c16"
Jul 17 06:22:52.132646 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:22:52.132619    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-dpsqq_odh-ai-gateway-infra(e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4"
Jul 17 06:22:53.162011 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:53.161980    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"]
Jul 17 06:22:53.299929 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:53.299907    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"
Jul 17 06:22:53.434869 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:53.434803    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4-maas-api-service-ca\") pod \"e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4\" (UID: \"e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4\") "
Jul 17 06:22:53.435130 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:53.435107    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" (UID: "e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:22:53.536350 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:53.536324    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:22:54.145663 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:54.145634    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"
Jul 17 06:22:54.145803 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:54.145671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq" event={"ID":"e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4","Type":"ContainerDied","Data":"a5dfd2a1be589541448e18434c41869f44edc96d4fd7951558444ac49f38c86b"}
Jul 17 06:22:54.145803 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:54.145734    2558 scope.go:117] "RemoveContainer" containerID="722f5e7ecfcc0ef09909696d768100113460246cb381a5460ddbc1af07634c16"
Jul 17 06:22:54.162615 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:54.162594    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"]
Jul 17 06:22:54.164554 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:54.164534    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-dpsqq"]
Jul 17 06:22:56.078288 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:56.078251    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" path="/var/lib/kubelet/pods/e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4/volumes"
Jul 17 06:22:57.171848 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.171811    2558 generic.go:356] "Generic (PLEG): container finished" podID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerID="c3eee1c56c89cf3ede38f2a412d5e8642c80d640bfaf66692c0f443945436d5c" exitCode=0
Jul 17 06:22:57.172268 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.171878    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" event={"ID":"892ae5cf-3368-4feb-84a9-2b81f45330cb","Type":"ContainerDied","Data":"c3eee1c56c89cf3ede38f2a412d5e8642c80d640bfaf66692c0f443945436d5c"}
Jul 17 06:22:57.172268 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.171940    2558 scope.go:117] "RemoveContainer" containerID="bf97542104fb9b7534afdf810652b47d15d4f19a0e7b496f71f08bf8c471f18a"
Jul 17 06:22:57.280365 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.280295    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:22:57.370127 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370097    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/892ae5cf-3368-4feb-84a9-2b81f45330cb-istiod-ca-cert\") pod \"892ae5cf-3368-4feb-84a9-2b81f45330cb\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") "
Jul 17 06:22:57.370310 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370160    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-credential-socket\") pod \"892ae5cf-3368-4feb-84a9-2b81f45330cb\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") "
Jul 17 06:22:57.370310 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370187    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-podinfo\") pod \"892ae5cf-3368-4feb-84a9-2b81f45330cb\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") "
Jul 17 06:22:57.370310 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370226    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jhms\" (UniqueName: \"kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-kube-api-access-6jhms\") pod \"892ae5cf-3368-4feb-84a9-2b81f45330cb\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") "
Jul 17 06:22:57.370310 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370280    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-certs\") pod \"892ae5cf-3368-4feb-84a9-2b81f45330cb\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") "
Jul 17 06:22:57.370547 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370318    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-socket\") pod \"892ae5cf-3368-4feb-84a9-2b81f45330cb\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") "
Jul 17 06:22:57.370547 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370343    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-data\") pod \"892ae5cf-3368-4feb-84a9-2b81f45330cb\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") "
Jul 17 06:22:57.370547 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370398    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-envoy\") pod \"892ae5cf-3368-4feb-84a9-2b81f45330cb\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") "
Jul 17 06:22:57.370547 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370430    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-token\") pod \"892ae5cf-3368-4feb-84a9-2b81f45330cb\" (UID: \"892ae5cf-3368-4feb-84a9-2b81f45330cb\") "
Jul 17 06:22:57.370547 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370522    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "892ae5cf-3368-4feb-84a9-2b81f45330cb" (UID: "892ae5cf-3368-4feb-84a9-2b81f45330cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:22:57.370878 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370622    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/892ae5cf-3368-4feb-84a9-2b81f45330cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "892ae5cf-3368-4feb-84a9-2b81f45330cb" (UID: "892ae5cf-3368-4feb-84a9-2b81f45330cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:22:57.370971 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370943    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/892ae5cf-3368-4feb-84a9-2b81f45330cb-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:22:57.371046 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.370971    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:22:57.371531 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.371492    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "892ae5cf-3368-4feb-84a9-2b81f45330cb" (UID: "892ae5cf-3368-4feb-84a9-2b81f45330cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:22:57.371659 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.371549    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "892ae5cf-3368-4feb-84a9-2b81f45330cb" (UID: "892ae5cf-3368-4feb-84a9-2b81f45330cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:22:57.371841 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.371813    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "892ae5cf-3368-4feb-84a9-2b81f45330cb" (UID: "892ae5cf-3368-4feb-84a9-2b81f45330cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:22:57.372896 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.372868    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "892ae5cf-3368-4feb-84a9-2b81f45330cb" (UID: "892ae5cf-3368-4feb-84a9-2b81f45330cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:22:57.372999 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.372980    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "892ae5cf-3368-4feb-84a9-2b81f45330cb" (UID: "892ae5cf-3368-4feb-84a9-2b81f45330cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:22:57.373182 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.373157    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "892ae5cf-3368-4feb-84a9-2b81f45330cb" (UID: "892ae5cf-3368-4feb-84a9-2b81f45330cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:22:57.373305 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.373212    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-kube-api-access-6jhms" (OuterVolumeSpecName: "kube-api-access-6jhms") pod "892ae5cf-3368-4feb-84a9-2b81f45330cb" (UID: "892ae5cf-3368-4feb-84a9-2b81f45330cb"). InnerVolumeSpecName "kube-api-access-6jhms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:22:57.472428 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.472381    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jhms\" (UniqueName: \"kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-kube-api-access-6jhms\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:22:57.472428 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.472419    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:22:57.472428 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.472430    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:22:57.472428 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.472438    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:22:57.472428 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.472447    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:22:57.472762 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.472454    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:22:57.472762 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:57.472462    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/892ae5cf-3368-4feb-84a9-2b81f45330cb-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:22:58.178037 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:58.178010    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"
Jul 17 06:22:58.178403 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:58.178010    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs" event={"ID":"892ae5cf-3368-4feb-84a9-2b81f45330cb","Type":"ContainerDied","Data":"20daa7883452aaa1210bc397934ba489b10a456bdcbf28bfe11062a84d02117a"}
Jul 17 06:22:58.178403 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:58.178146    2558 scope.go:117] "RemoveContainer" containerID="c3eee1c56c89cf3ede38f2a412d5e8642c80d640bfaf66692c0f443945436d5c"
Jul 17 06:22:58.197373 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:58.197344    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"]
Jul 17 06:22:58.201615 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:22:58.201591    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-572ab047-openshift-default-594468b699-r84cs"]
Jul 17 06:23:00.079163 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:00.079125    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" path="/var/lib/kubelet/pods/892ae5cf-3368-4feb-84a9-2b81f45330cb/volumes"
Jul 17 06:23:04.071118 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071079    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"]
Jul 17 06:23:04.071653 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071634    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerName="revoke-keys"
Jul 17 06:23:04.071720 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071656    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5070c26-0351-4080-9d41-fb60c387b2b4" containerName="revoke-keys"
Jul 17 06:23:04.071720 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071668    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerName="revoke-keys"
Jul 17 06:23:04.071720 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071677    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerName="revoke-keys"
Jul 17 06:23:04.071720 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071694    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerName="revoke-keys"
Jul 17 06:23:04.071720 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071700    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerName="revoke-keys"
Jul 17 06:23:04.071720 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071707    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerName="revoke-keys"
Jul 17 06:23:04.071720 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071715    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerName="revoke-keys"
Jul 17 06:23:04.072022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071741    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerName="istio-proxy"
Jul 17 06:23:04.072022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071746    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerName="istio-proxy"
Jul 17 06:23:04.072022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071755    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerName="istio-proxy"
Jul 17 06:23:04.072022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071760    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerName="istio-proxy"
Jul 17 06:23:04.072022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071829    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerName="revoke-keys"
Jul 17 06:23:04.072022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071839    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerName="istio-proxy"
Jul 17 06:23:04.072022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071845    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="892ae5cf-3368-4feb-84a9-2b81f45330cb" containerName="istio-proxy"
Jul 17 06:23:04.072022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.071852    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerName="revoke-keys"
Jul 17 06:23:04.076367 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.076344    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.080385 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.080364    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-51e238b9-openshift-default-dockercfg-j7rz5\""
Jul 17 06:23:04.115017 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.114987    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"]
Jul 17 06:23:04.238187 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.238152    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-data\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.238187 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.238192    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istiod-ca-cert\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.238467 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.238242    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-certs\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.238467 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.238286    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-podinfo\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.238467 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.238340    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-socket\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.238467 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.238363    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zg62\" (UniqueName: \"kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-kube-api-access-7zg62\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.238467 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.238438    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-envoy\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.238467 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.238513    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-token\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.238758 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.238570    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-credential-socket\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.339962 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.339930    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-certs\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340146 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.339988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-podinfo\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340146 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.340045    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-socket\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340146 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.340071    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zg62\" (UniqueName: \"kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-kube-api-access-7zg62\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340146 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.340129    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-envoy\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340399 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.340155    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-token\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340399 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.340196    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-credential-socket\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340399 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.340236    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-data\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340399 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.340266    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istiod-ca-cert\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340399 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.340351    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-certs\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340720 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.340425    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-socket\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340782 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.340731    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-credential-socket\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.340909 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.340887    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-data\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.341231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.341200    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istiod-ca-cert\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.342768 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.342735    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-envoy\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.342957 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.342932    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-podinfo\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.349020 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.348980    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-token\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.349128 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.348986    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zg62\" (UniqueName: \"kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-kube-api-access-7zg62\") pod \"e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") " pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.389127 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.389084    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:04.545430 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.545398    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"]
Jul 17 06:23:04.546650 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:23:04.546622    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a45d1c_f206_4f8e_a24d_42fe3dc471bf.slice/crio-489537f15a1815aa64ebf94c80aaeb602ed1b332a77db1ec6eef9fc6235fdf23 WatchSource:0}: Error finding container 489537f15a1815aa64ebf94c80aaeb602ed1b332a77db1ec6eef9fc6235fdf23: Status 404 returned error can't find the container with id 489537f15a1815aa64ebf94c80aaeb602ed1b332a77db1ec6eef9fc6235fdf23
Jul 17 06:23:04.548929 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.548892    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:23:04.549017 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.548967    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:23:04.549017 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:04.548996    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:23:05.225599 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:05.225563    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" event={"ID":"02a45d1c-f206-4f8e-a24d-42fe3dc471bf","Type":"ContainerStarted","Data":"eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f"}
Jul 17 06:23:05.225987 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:05.225618    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" event={"ID":"02a45d1c-f206-4f8e-a24d-42fe3dc471bf","Type":"ContainerStarted","Data":"489537f15a1815aa64ebf94c80aaeb602ed1b332a77db1ec6eef9fc6235fdf23"}
Jul 17 06:23:05.253647 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:05.248086    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" podStartSLOduration=1.248067284 podStartE2EDuration="1.248067284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:23:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:23:05.245450889 +0000 UTC m=+2507.687389479" watchObservedRunningTime="2026-07-17 06:23:05.248067284 +0000 UTC m=+2507.690005859"
Jul 17 06:23:05.390153 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:05.390121    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:05.394939 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:05.394917    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:06.232196 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:06.232154    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:06.233316 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:06.233297    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:16.292295 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:16.292265    2558 generic.go:356] "Generic (PLEG): container finished" podID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerID="eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f" exitCode=0
Jul 17 06:23:16.292698 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:16.292326    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" event={"ID":"02a45d1c-f206-4f8e-a24d-42fe3dc471bf","Type":"ContainerDied","Data":"eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f"}
Jul 17 06:23:16.292698 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:16.292687    2558 scope.go:117] "RemoveContainer" containerID="eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f"
Jul 17 06:23:16.293694 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:16.293666    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:23:16.293764 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:16.293750    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:23:16.293803 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:16.293790    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:23:17.300112 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:17.300081    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" event={"ID":"02a45d1c-f206-4f8e-a24d-42fe3dc471bf","Type":"ContainerStarted","Data":"5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9"}
Jul 17 06:23:17.390223 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:17.390189    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:17.390943 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:17.390922    2558 patch_prober.go:28] interesting pod/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 17 06:23:17.391003 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:17.390975    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 17 06:23:18.390043 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:18.390012    2558 patch_prober.go:28] interesting pod/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 17 06:23:18.390414 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:18.390075    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 17 06:23:19.390145 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:19.390112    2558 patch_prober.go:28] interesting pod/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 17 06:23:19.390560 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:19.390180    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 17 06:23:20.390464 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:20.390426    2558 patch_prober.go:28] interesting pod/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 17 06:23:20.390888 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:20.390527    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 17 06:23:21.232140 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:21.232104    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:21.394545 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:21.394512    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:21.395560 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:21.395542    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:23:23.686799 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:23.686765    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"]
Jul 17 06:23:23.687265 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:23.687252    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e697cdc0-65fd-46f7-9a15-7fbbd5ee9dc4" containerName="revoke-keys"
Jul 17 06:23:23.690543 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:23.690527    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"
Jul 17 06:23:23.693683 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:23.693668    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:23:23.700815 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:23.700791    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"]
Jul 17 06:23:23.723414 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:23.723378    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c481f31-3892-434e-b3a0-bf2573e5e0db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz\" (UID: \"5c481f31-3892-434e-b3a0-bf2573e5e0db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"
Jul 17 06:23:23.824198 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:23.824162    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c481f31-3892-434e-b3a0-bf2573e5e0db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz\" (UID: \"5c481f31-3892-434e-b3a0-bf2573e5e0db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"
Jul 17 06:23:23.824887 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:23.824859    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c481f31-3892-434e-b3a0-bf2573e5e0db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz\" (UID: \"5c481f31-3892-434e-b3a0-bf2573e5e0db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"
Jul 17 06:23:24.002978 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:24.002893    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"
Jul 17 06:23:24.140901 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:24.140880    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"]
Jul 17 06:23:24.143868 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:23:24.143838    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c481f31_3892_434e_b3a0_bf2573e5e0db.slice/crio-b3e0a0835e4f0b4e3de5530f942e13e63877eed7fbb7a8734e0dee93a68fe1e4 WatchSource:0}: Error finding container b3e0a0835e4f0b4e3de5530f942e13e63877eed7fbb7a8734e0dee93a68fe1e4: Status 404 returned error can't find the container with id b3e0a0835e4f0b4e3de5530f942e13e63877eed7fbb7a8734e0dee93a68fe1e4
Jul 17 06:23:24.338437 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:24.338403    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz" event={"ID":"5c481f31-3892-434e-b3a0-bf2573e5e0db","Type":"ContainerStarted","Data":"b3e0a0835e4f0b4e3de5530f942e13e63877eed7fbb7a8734e0dee93a68fe1e4"}
Jul 17 06:23:25.346060 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:25.346030    2558 generic.go:356] "Generic (PLEG): container finished" podID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerID="429cb4f96a28cb24ade91f6bb663e2f53610eb7ef0c1d5d057c6b2c9ee481fd9" exitCode=6
Jul 17 06:23:25.346492 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:25.346125    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz" event={"ID":"5c481f31-3892-434e-b3a0-bf2573e5e0db","Type":"ContainerDied","Data":"429cb4f96a28cb24ade91f6bb663e2f53610eb7ef0c1d5d057c6b2c9ee481fd9"}
Jul 17 06:23:25.346492 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:25.346276    2558 scope.go:117] "RemoveContainer" containerID="429cb4f96a28cb24ade91f6bb663e2f53610eb7ef0c1d5d057c6b2c9ee481fd9"
Jul 17 06:23:26.352906 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:26.352867    2558 generic.go:356] "Generic (PLEG): container finished" podID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerID="de8de90c4895a71053109aa96afb8af2e27f3658c841ebf6a1bfa20865d7cef1" exitCode=6
Jul 17 06:23:26.353376 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:26.352951    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz" event={"ID":"5c481f31-3892-434e-b3a0-bf2573e5e0db","Type":"ContainerDied","Data":"de8de90c4895a71053109aa96afb8af2e27f3658c841ebf6a1bfa20865d7cef1"}
Jul 17 06:23:26.353376 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:26.353029    2558 scope.go:117] "RemoveContainer" containerID="429cb4f96a28cb24ade91f6bb663e2f53610eb7ef0c1d5d057c6b2c9ee481fd9"
Jul 17 06:23:26.353376 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:26.353330    2558 scope.go:117] "RemoveContainer" containerID="de8de90c4895a71053109aa96afb8af2e27f3658c841ebf6a1bfa20865d7cef1"
Jul 17 06:23:26.353607 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:23:26.353587    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz_odh-ai-gateway-infra(5c481f31-3892-434e-b3a0-bf2573e5e0db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db"
Jul 17 06:23:27.360879 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:27.360853    2558 scope.go:117] "RemoveContainer" containerID="de8de90c4895a71053109aa96afb8af2e27f3658c841ebf6a1bfa20865d7cef1"
Jul 17 06:23:27.361291 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:23:27.361071    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz_odh-ai-gateway-infra(5c481f31-3892-434e-b3a0-bf2573e5e0db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db"
Jul 17 06:23:42.079991 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:42.079962    2558 scope.go:117] "RemoveContainer" containerID="de8de90c4895a71053109aa96afb8af2e27f3658c841ebf6a1bfa20865d7cef1"
Jul 17 06:23:42.441439 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:42.441403    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz" event={"ID":"5c481f31-3892-434e-b3a0-bf2573e5e0db","Type":"ContainerStarted","Data":"8b72bfe1a18ecab33e050064e9cc0ee8151c5f226218ffae52f63be33ac0ce76"}
Jul 17 06:23:42.460127 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:42.460074    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz" podStartSLOduration=19.46005983 podStartE2EDuration="19.46005983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:23:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:23:42.458201314 +0000 UTC m=+2544.900139894" watchObservedRunningTime="2026-07-17 06:23:42.46005983 +0000 UTC m=+2544.901998404"
Jul 17 06:23:43.455779 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:43.455740    2558 generic.go:356] "Generic (PLEG): container finished" podID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerID="8b72bfe1a18ecab33e050064e9cc0ee8151c5f226218ffae52f63be33ac0ce76" exitCode=6
Jul 17 06:23:43.456233 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:43.455806    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz" event={"ID":"5c481f31-3892-434e-b3a0-bf2573e5e0db","Type":"ContainerDied","Data":"8b72bfe1a18ecab33e050064e9cc0ee8151c5f226218ffae52f63be33ac0ce76"}
Jul 17 06:23:43.456233 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:43.455891    2558 scope.go:117] "RemoveContainer" containerID="de8de90c4895a71053109aa96afb8af2e27f3658c841ebf6a1bfa20865d7cef1"
Jul 17 06:23:43.456233 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:43.456141    2558 scope.go:117] "RemoveContainer" containerID="8b72bfe1a18ecab33e050064e9cc0ee8151c5f226218ffae52f63be33ac0ce76"
Jul 17 06:23:43.456379 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:23:43.456362    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz_odh-ai-gateway-infra(5c481f31-3892-434e-b3a0-bf2573e5e0db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db"
Jul 17 06:23:43.470720 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:43.470700    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"]
Jul 17 06:23:44.615820 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:44.615790    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"
Jul 17 06:23:44.727703 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:44.727598    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c481f31-3892-434e-b3a0-bf2573e5e0db-maas-api-service-ca\") pod \"5c481f31-3892-434e-b3a0-bf2573e5e0db\" (UID: \"5c481f31-3892-434e-b3a0-bf2573e5e0db\") "
Jul 17 06:23:44.727962 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:44.727940    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c481f31-3892-434e-b3a0-bf2573e5e0db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5c481f31-3892-434e-b3a0-bf2573e5e0db" (UID: "5c481f31-3892-434e-b3a0-bf2573e5e0db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:23:44.829044 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:44.828988    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c481f31-3892-434e-b3a0-bf2573e5e0db-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:23:45.468702 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.468674    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"
Jul 17 06:23:45.468898 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.468662    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz" event={"ID":"5c481f31-3892-434e-b3a0-bf2573e5e0db","Type":"ContainerDied","Data":"b3e0a0835e4f0b4e3de5530f942e13e63877eed7fbb7a8734e0dee93a68fe1e4"}
Jul 17 06:23:45.468898 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.468822    2558 scope.go:117] "RemoveContainer" containerID="8b72bfe1a18ecab33e050064e9cc0ee8151c5f226218ffae52f63be33ac0ce76"
Jul 17 06:23:45.496908 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.496868    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"]
Jul 17 06:23:45.507634 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.507601    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-l6lbz"]
Jul 17 06:23:45.852352 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.852316    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"]
Jul 17 06:23:45.852833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.852753    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerName="revoke-keys"
Jul 17 06:23:45.852833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.852765    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerName="revoke-keys"
Jul 17 06:23:45.852833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.852780    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerName="revoke-keys"
Jul 17 06:23:45.852833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.852785    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerName="revoke-keys"
Jul 17 06:23:45.852833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.852797    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerName="revoke-keys"
Jul 17 06:23:45.852833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.852803    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerName="revoke-keys"
Jul 17 06:23:45.853062 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.852869    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerName="revoke-keys"
Jul 17 06:23:45.853062 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.852876    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerName="revoke-keys"
Jul 17 06:23:45.857561 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.857543    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"
Jul 17 06:23:45.861378 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.861356    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:23:45.870881 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.870859    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"]
Jul 17 06:23:45.940460 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:45.940422    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8818f6c1-ac29-458c-ba98-672dc3280521-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-gt68v\" (UID: \"8818f6c1-ac29-458c-ba98-672dc3280521\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"
Jul 17 06:23:46.041372 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:46.041316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8818f6c1-ac29-458c-ba98-672dc3280521-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-gt68v\" (UID: \"8818f6c1-ac29-458c-ba98-672dc3280521\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"
Jul 17 06:23:46.042094 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:46.042075    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8818f6c1-ac29-458c-ba98-672dc3280521-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-gt68v\" (UID: \"8818f6c1-ac29-458c-ba98-672dc3280521\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"
Jul 17 06:23:46.078767 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:46.078735    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db" path="/var/lib/kubelet/pods/5c481f31-3892-434e-b3a0-bf2573e5e0db/volumes"
Jul 17 06:23:46.167876 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:46.167795    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"
Jul 17 06:23:46.312657 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:46.312634    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"]
Jul 17 06:23:46.314960 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:23:46.314932    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8818f6c1_ac29_458c_ba98_672dc3280521.slice/crio-45cfdd69142374dc8c88d241720a00b2f01a9313424f3e7fe849ba069afe0021 WatchSource:0}: Error finding container 45cfdd69142374dc8c88d241720a00b2f01a9313424f3e7fe849ba069afe0021: Status 404 returned error can't find the container with id 45cfdd69142374dc8c88d241720a00b2f01a9313424f3e7fe849ba069afe0021
Jul 17 06:23:46.476748 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:46.476653    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v" event={"ID":"8818f6c1-ac29-458c-ba98-672dc3280521","Type":"ContainerStarted","Data":"45cfdd69142374dc8c88d241720a00b2f01a9313424f3e7fe849ba069afe0021"}
Jul 17 06:23:47.484230 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:47.484195    2558 generic.go:356] "Generic (PLEG): container finished" podID="8818f6c1-ac29-458c-ba98-672dc3280521" containerID="8826ac681938d7d89f5e72db9ee3fd5ebb3d1c9277438a4e422a1ebc8ebb1093" exitCode=6
Jul 17 06:23:47.484624 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:47.484272    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v" event={"ID":"8818f6c1-ac29-458c-ba98-672dc3280521","Type":"ContainerDied","Data":"8826ac681938d7d89f5e72db9ee3fd5ebb3d1c9277438a4e422a1ebc8ebb1093"}
Jul 17 06:23:47.484624 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:47.484482    2558 scope.go:117] "RemoveContainer" containerID="8826ac681938d7d89f5e72db9ee3fd5ebb3d1c9277438a4e422a1ebc8ebb1093"
Jul 17 06:23:48.492207 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:48.492170    2558 generic.go:356] "Generic (PLEG): container finished" podID="8818f6c1-ac29-458c-ba98-672dc3280521" containerID="b3a2db3feb3256a661c5bbacdfaba9eb8e5d7aa743d8cfb3eec017902ccb9a53" exitCode=6
Jul 17 06:23:48.492721 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:48.492253    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v" event={"ID":"8818f6c1-ac29-458c-ba98-672dc3280521","Type":"ContainerDied","Data":"b3a2db3feb3256a661c5bbacdfaba9eb8e5d7aa743d8cfb3eec017902ccb9a53"}
Jul 17 06:23:48.492721 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:48.492313    2558 scope.go:117] "RemoveContainer" containerID="8826ac681938d7d89f5e72db9ee3fd5ebb3d1c9277438a4e422a1ebc8ebb1093"
Jul 17 06:23:48.492721 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:48.492623    2558 scope.go:117] "RemoveContainer" containerID="b3a2db3feb3256a661c5bbacdfaba9eb8e5d7aa743d8cfb3eec017902ccb9a53"
Jul 17 06:23:48.492943 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:23:48.492917    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-gt68v_odh-ai-gateway-infra(8818f6c1-ac29-458c-ba98-672dc3280521)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v" podUID="8818f6c1-ac29-458c-ba98-672dc3280521"
Jul 17 06:23:49.501488 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:23:49.501456    2558 scope.go:117] "RemoveContainer" containerID="b3a2db3feb3256a661c5bbacdfaba9eb8e5d7aa743d8cfb3eec017902ccb9a53"
Jul 17 06:23:49.501895 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:23:49.501729    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-gt68v_odh-ai-gateway-infra(8818f6c1-ac29-458c-ba98-672dc3280521)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v" podUID="8818f6c1-ac29-458c-ba98-672dc3280521"
Jul 17 06:24:02.072766 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:02.072731    2558 scope.go:117] "RemoveContainer" containerID="b3a2db3feb3256a661c5bbacdfaba9eb8e5d7aa743d8cfb3eec017902ccb9a53"
Jul 17 06:24:02.573580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:02.573543    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v" event={"ID":"8818f6c1-ac29-458c-ba98-672dc3280521","Type":"ContainerStarted","Data":"5b75f8cd87839343c54e73a92001f01361f2f003fa678566ade2dadbc8f2b4cb"}
Jul 17 06:24:02.591145 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:02.591104    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v" podStartSLOduration=17.591091028 podStartE2EDuration="17.591091028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:23:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:24:02.589263737 +0000 UTC m=+2565.031202313" watchObservedRunningTime="2026-07-17 06:24:02.591091028 +0000 UTC m=+2565.033029603"
Jul 17 06:24:03.581055 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:03.581023    2558 generic.go:356] "Generic (PLEG): container finished" podID="8818f6c1-ac29-458c-ba98-672dc3280521" containerID="5b75f8cd87839343c54e73a92001f01361f2f003fa678566ade2dadbc8f2b4cb" exitCode=6
Jul 17 06:24:03.581539 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:03.581099    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v" event={"ID":"8818f6c1-ac29-458c-ba98-672dc3280521","Type":"ContainerDied","Data":"5b75f8cd87839343c54e73a92001f01361f2f003fa678566ade2dadbc8f2b4cb"}
Jul 17 06:24:03.581539 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:03.581162    2558 scope.go:117] "RemoveContainer" containerID="b3a2db3feb3256a661c5bbacdfaba9eb8e5d7aa743d8cfb3eec017902ccb9a53"
Jul 17 06:24:03.581539 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:03.581504    2558 scope.go:117] "RemoveContainer" containerID="5b75f8cd87839343c54e73a92001f01361f2f003fa678566ade2dadbc8f2b4cb"
Jul 17 06:24:03.581755 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:24:03.581703    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-gt68v_odh-ai-gateway-infra(8818f6c1-ac29-458c-ba98-672dc3280521)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v" podUID="8818f6c1-ac29-458c-ba98-672dc3280521"
Jul 17 06:24:03.604155 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:03.604130    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"]
Jul 17 06:24:04.739173 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:04.739151    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"
Jul 17 06:24:04.900835 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:04.900768    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8818f6c1-ac29-458c-ba98-672dc3280521-maas-api-service-ca\") pod \"8818f6c1-ac29-458c-ba98-672dc3280521\" (UID: \"8818f6c1-ac29-458c-ba98-672dc3280521\") "
Jul 17 06:24:04.901149 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:04.901127    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8818f6c1-ac29-458c-ba98-672dc3280521-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8818f6c1-ac29-458c-ba98-672dc3280521" (UID: "8818f6c1-ac29-458c-ba98-672dc3280521"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:24:05.001965 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:05.001939    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8818f6c1-ac29-458c-ba98-672dc3280521-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:24:05.596854 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:05.596818    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"
Jul 17 06:24:05.597062 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:05.596848    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v" event={"ID":"8818f6c1-ac29-458c-ba98-672dc3280521","Type":"ContainerDied","Data":"45cfdd69142374dc8c88d241720a00b2f01a9313424f3e7fe849ba069afe0021"}
Jul 17 06:24:05.597062 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:05.596927    2558 scope.go:117] "RemoveContainer" containerID="5b75f8cd87839343c54e73a92001f01361f2f003fa678566ade2dadbc8f2b4cb"
Jul 17 06:24:05.620739 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:05.620708    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"]
Jul 17 06:24:05.626223 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:05.626197    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-gt68v"]
Jul 17 06:24:06.078376 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:06.078292    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8818f6c1-ac29-458c-ba98-672dc3280521" path="/var/lib/kubelet/pods/8818f6c1-ac29-458c-ba98-672dc3280521/volumes"
Jul 17 06:24:09.737997 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.737963    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"]
Jul 17 06:24:09.738659 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.738639    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8818f6c1-ac29-458c-ba98-672dc3280521" containerName="revoke-keys"
Jul 17 06:24:09.738725 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.738662    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8818f6c1-ac29-458c-ba98-672dc3280521" containerName="revoke-keys"
Jul 17 06:24:09.738725 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.738682    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8818f6c1-ac29-458c-ba98-672dc3280521" containerName="revoke-keys"
Jul 17 06:24:09.738725 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.738693    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8818f6c1-ac29-458c-ba98-672dc3280521" containerName="revoke-keys"
Jul 17 06:24:09.738725 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.738716    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8818f6c1-ac29-458c-ba98-672dc3280521" containerName="revoke-keys"
Jul 17 06:24:09.738725 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.738725    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8818f6c1-ac29-458c-ba98-672dc3280521" containerName="revoke-keys"
Jul 17 06:24:09.738885 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.738812    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c481f31-3892-434e-b3a0-bf2573e5e0db" containerName="revoke-keys"
Jul 17 06:24:09.738885 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.738825    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8818f6c1-ac29-458c-ba98-672dc3280521" containerName="revoke-keys"
Jul 17 06:24:09.738885 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.738834    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8818f6c1-ac29-458c-ba98-672dc3280521" containerName="revoke-keys"
Jul 17 06:24:09.738885 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.738845    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8818f6c1-ac29-458c-ba98-672dc3280521" containerName="revoke-keys"
Jul 17 06:24:09.742273 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.742255    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"
Jul 17 06:24:09.745261 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.745240    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:24:09.749422 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.749396    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"]
Jul 17 06:24:09.843507 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.843461    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3a9e1c-f840-40d9-bd21-e949e055c29c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-m72j5\" (UID: \"ae3a9e1c-f840-40d9-bd21-e949e055c29c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"
Jul 17 06:24:09.944775 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.944744    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3a9e1c-f840-40d9-bd21-e949e055c29c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-m72j5\" (UID: \"ae3a9e1c-f840-40d9-bd21-e949e055c29c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"
Jul 17 06:24:09.945352 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:09.945327    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3a9e1c-f840-40d9-bd21-e949e055c29c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-m72j5\" (UID: \"ae3a9e1c-f840-40d9-bd21-e949e055c29c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"
Jul 17 06:24:10.056983 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:10.056927    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"
Jul 17 06:24:10.415808 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:10.415779    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"]
Jul 17 06:24:10.417233 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:24:10.417208    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3a9e1c_f840_40d9_bd21_e949e055c29c.slice/crio-90c65a4104fdd559f20b787401a8f854c26c0439abefb1afda98445a303e67a0 WatchSource:0}: Error finding container 90c65a4104fdd559f20b787401a8f854c26c0439abefb1afda98445a303e67a0: Status 404 returned error can't find the container with id 90c65a4104fdd559f20b787401a8f854c26c0439abefb1afda98445a303e67a0
Jul 17 06:24:10.626285 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:10.626250    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5" event={"ID":"ae3a9e1c-f840-40d9-bd21-e949e055c29c","Type":"ContainerStarted","Data":"90c65a4104fdd559f20b787401a8f854c26c0439abefb1afda98445a303e67a0"}
Jul 17 06:24:11.633293 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:11.633254    2558 generic.go:356] "Generic (PLEG): container finished" podID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerID="8b906c6858c2a2e19ea2c4135895ce339282cd6940e21d4a23f6cd05ff5e931c" exitCode=7
Jul 17 06:24:11.633711 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:11.633350    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5" event={"ID":"ae3a9e1c-f840-40d9-bd21-e949e055c29c","Type":"ContainerDied","Data":"8b906c6858c2a2e19ea2c4135895ce339282cd6940e21d4a23f6cd05ff5e931c"}
Jul 17 06:24:11.633711 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:11.633518    2558 scope.go:117] "RemoveContainer" containerID="8b906c6858c2a2e19ea2c4135895ce339282cd6940e21d4a23f6cd05ff5e931c"
Jul 17 06:24:12.641436 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:12.641407    2558 generic.go:356] "Generic (PLEG): container finished" podID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerID="9e9c781c2e5ad6ca2202b3074a23153f9b87aba04bbcca7aabcff87ea92325a9" exitCode=7
Jul 17 06:24:12.641905 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:12.641500    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5" event={"ID":"ae3a9e1c-f840-40d9-bd21-e949e055c29c","Type":"ContainerDied","Data":"9e9c781c2e5ad6ca2202b3074a23153f9b87aba04bbcca7aabcff87ea92325a9"}
Jul 17 06:24:12.641905 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:12.641557    2558 scope.go:117] "RemoveContainer" containerID="8b906c6858c2a2e19ea2c4135895ce339282cd6940e21d4a23f6cd05ff5e931c"
Jul 17 06:24:12.641905 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:12.641706    2558 scope.go:117] "RemoveContainer" containerID="9e9c781c2e5ad6ca2202b3074a23153f9b87aba04bbcca7aabcff87ea92325a9"
Jul 17 06:24:12.642081 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:24:12.641928    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-m72j5_odh-ai-gateway-infra(ae3a9e1c-f840-40d9-bd21-e949e055c29c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c"
Jul 17 06:24:13.649700 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:13.649672    2558 scope.go:117] "RemoveContainer" containerID="9e9c781c2e5ad6ca2202b3074a23153f9b87aba04bbcca7aabcff87ea92325a9"
Jul 17 06:24:13.650044 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:24:13.649853    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-m72j5_odh-ai-gateway-infra(ae3a9e1c-f840-40d9-bd21-e949e055c29c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c"
Jul 17 06:24:22.179675 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:22.179633    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"]
Jul 17 06:24:22.184873 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:22.184847    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"
Jul 17 06:24:22.196967 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:22.194198    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"]
Jul 17 06:24:22.248370 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:22.248321    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1edee576-1c31-4ccf-8af4-ebf70c57828b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-s2tc9\" (UID: \"1edee576-1c31-4ccf-8af4-ebf70c57828b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"
Jul 17 06:24:22.349680 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:22.349577    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1edee576-1c31-4ccf-8af4-ebf70c57828b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-s2tc9\" (UID: \"1edee576-1c31-4ccf-8af4-ebf70c57828b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"
Jul 17 06:24:22.350237 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:22.350213    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1edee576-1c31-4ccf-8af4-ebf70c57828b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-s2tc9\" (UID: \"1edee576-1c31-4ccf-8af4-ebf70c57828b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"
Jul 17 06:24:22.501920 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:22.501878    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"
Jul 17 06:24:22.648148 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:22.648121    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"]
Jul 17 06:24:22.650276 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:24:22.650249    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edee576_1c31_4ccf_8af4_ebf70c57828b.slice/crio-70c818c47dc016ce71fcccba57cd82163d11f56c9f3bb092c6ef8d6c4edfbb96 WatchSource:0}: Error finding container 70c818c47dc016ce71fcccba57cd82163d11f56c9f3bb092c6ef8d6c4edfbb96: Status 404 returned error can't find the container with id 70c818c47dc016ce71fcccba57cd82163d11f56c9f3bb092c6ef8d6c4edfbb96
Jul 17 06:24:22.700062 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:22.700030    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9" event={"ID":"1edee576-1c31-4ccf-8af4-ebf70c57828b","Type":"ContainerStarted","Data":"70c818c47dc016ce71fcccba57cd82163d11f56c9f3bb092c6ef8d6c4edfbb96"}
Jul 17 06:24:23.708075 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:23.708041    2558 generic.go:356] "Generic (PLEG): container finished" podID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerID="7712b03b6d525213d30f14bcb83d8b2427d809aca16464a56e93a47328505b34" exitCode=6
Jul 17 06:24:23.708675 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:23.708122    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9" event={"ID":"1edee576-1c31-4ccf-8af4-ebf70c57828b","Type":"ContainerDied","Data":"7712b03b6d525213d30f14bcb83d8b2427d809aca16464a56e93a47328505b34"}
Jul 17 06:24:23.708675 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:23.708331    2558 scope.go:117] "RemoveContainer" containerID="7712b03b6d525213d30f14bcb83d8b2427d809aca16464a56e93a47328505b34"
Jul 17 06:24:24.715061 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:24.715015    2558 generic.go:356] "Generic (PLEG): container finished" podID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerID="cdeca3b6c8cf314640048de37d3aad6d2478d1b881a00d31cc41b90b661a34f1" exitCode=6
Jul 17 06:24:24.715499 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:24.715106    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9" event={"ID":"1edee576-1c31-4ccf-8af4-ebf70c57828b","Type":"ContainerDied","Data":"cdeca3b6c8cf314640048de37d3aad6d2478d1b881a00d31cc41b90b661a34f1"}
Jul 17 06:24:24.715499 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:24.715178    2558 scope.go:117] "RemoveContainer" containerID="7712b03b6d525213d30f14bcb83d8b2427d809aca16464a56e93a47328505b34"
Jul 17 06:24:24.715499 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:24.715465    2558 scope.go:117] "RemoveContainer" containerID="cdeca3b6c8cf314640048de37d3aad6d2478d1b881a00d31cc41b90b661a34f1"
Jul 17 06:24:24.715697 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:24:24.715679    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-s2tc9_odh-ai-gateway-infra(1edee576-1c31-4ccf-8af4-ebf70c57828b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b"
Jul 17 06:24:25.722266 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:25.722235    2558 scope.go:117] "RemoveContainer" containerID="cdeca3b6c8cf314640048de37d3aad6d2478d1b881a00d31cc41b90b661a34f1"
Jul 17 06:24:25.722725 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:24:25.722462    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-s2tc9_odh-ai-gateway-infra(1edee576-1c31-4ccf-8af4-ebf70c57828b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b"
Jul 17 06:24:26.072705 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:26.072619    2558 scope.go:117] "RemoveContainer" containerID="9e9c781c2e5ad6ca2202b3074a23153f9b87aba04bbcca7aabcff87ea92325a9"
Jul 17 06:24:26.730147 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:26.730110    2558 generic.go:356] "Generic (PLEG): container finished" podID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerID="a2e803ea06f339af7cd204a9d0061072337dc05eda5d94152407f0bf7d3d361c" exitCode=7
Jul 17 06:24:26.730611 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:26.730190    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5" event={"ID":"ae3a9e1c-f840-40d9-bd21-e949e055c29c","Type":"ContainerDied","Data":"a2e803ea06f339af7cd204a9d0061072337dc05eda5d94152407f0bf7d3d361c"}
Jul 17 06:24:26.730611 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:26.730275    2558 scope.go:117] "RemoveContainer" containerID="9e9c781c2e5ad6ca2202b3074a23153f9b87aba04bbcca7aabcff87ea92325a9"
Jul 17 06:24:26.730745 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:26.730655    2558 scope.go:117] "RemoveContainer" containerID="a2e803ea06f339af7cd204a9d0061072337dc05eda5d94152407f0bf7d3d361c"
Jul 17 06:24:26.730888 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:24:26.730867    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-m72j5_odh-ai-gateway-infra(ae3a9e1c-f840-40d9-bd21-e949e055c29c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c"
Jul 17 06:24:27.756910 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:27.756867    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"]
Jul 17 06:24:27.911398 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:27.911369    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"
Jul 17 06:24:28.004561 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:28.004518    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3a9e1c-f840-40d9-bd21-e949e055c29c-maas-api-service-ca\") pod \"ae3a9e1c-f840-40d9-bd21-e949e055c29c\" (UID: \"ae3a9e1c-f840-40d9-bd21-e949e055c29c\") "
Jul 17 06:24:28.004945 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:28.004921    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae3a9e1c-f840-40d9-bd21-e949e055c29c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ae3a9e1c-f840-40d9-bd21-e949e055c29c" (UID: "ae3a9e1c-f840-40d9-bd21-e949e055c29c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:24:28.105535 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:28.105497    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3a9e1c-f840-40d9-bd21-e949e055c29c-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:24:28.743332 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:28.743305    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"
Jul 17 06:24:28.743536 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:28.743332    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5" event={"ID":"ae3a9e1c-f840-40d9-bd21-e949e055c29c","Type":"ContainerDied","Data":"90c65a4104fdd559f20b787401a8f854c26c0439abefb1afda98445a303e67a0"}
Jul 17 06:24:28.743536 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:28.743383    2558 scope.go:117] "RemoveContainer" containerID="a2e803ea06f339af7cd204a9d0061072337dc05eda5d94152407f0bf7d3d361c"
Jul 17 06:24:28.760391 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:28.760366    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"]
Jul 17 06:24:28.769228 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:28.769205    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-m72j5"]
Jul 17 06:24:30.078804 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:30.078763    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" path="/var/lib/kubelet/pods/ae3a9e1c-f840-40d9-bd21-e949e055c29c/volumes"
Jul 17 06:24:41.072652 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:41.072573    2558 scope.go:117] "RemoveContainer" containerID="cdeca3b6c8cf314640048de37d3aad6d2478d1b881a00d31cc41b90b661a34f1"
Jul 17 06:24:41.816247 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:41.816218    2558 generic.go:356] "Generic (PLEG): container finished" podID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerID="e99296150bd0fbe98d168c40ae181c78770c3beef04e6c09e7f51618b34a7127" exitCode=6
Jul 17 06:24:41.816366 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:41.816291    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9" event={"ID":"1edee576-1c31-4ccf-8af4-ebf70c57828b","Type":"ContainerDied","Data":"e99296150bd0fbe98d168c40ae181c78770c3beef04e6c09e7f51618b34a7127"}
Jul 17 06:24:41.816366 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:41.816359    2558 scope.go:117] "RemoveContainer" containerID="cdeca3b6c8cf314640048de37d3aad6d2478d1b881a00d31cc41b90b661a34f1"
Jul 17 06:24:41.816612 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:41.816595    2558 scope.go:117] "RemoveContainer" containerID="e99296150bd0fbe98d168c40ae181c78770c3beef04e6c09e7f51618b34a7127"
Jul 17 06:24:41.816874 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:24:41.816844    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-s2tc9_odh-ai-gateway-infra(1edee576-1c31-4ccf-8af4-ebf70c57828b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b"
Jul 17 06:24:42.847433 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:42.847394    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"]
Jul 17 06:24:43.004680 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:43.004650    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"
Jul 17 06:24:43.137665 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:43.137564    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1edee576-1c31-4ccf-8af4-ebf70c57828b-maas-api-service-ca\") pod \"1edee576-1c31-4ccf-8af4-ebf70c57828b\" (UID: \"1edee576-1c31-4ccf-8af4-ebf70c57828b\") "
Jul 17 06:24:43.137959 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:43.137935    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1edee576-1c31-4ccf-8af4-ebf70c57828b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1edee576-1c31-4ccf-8af4-ebf70c57828b" (UID: "1edee576-1c31-4ccf-8af4-ebf70c57828b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:24:43.238580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:43.238538    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1edee576-1c31-4ccf-8af4-ebf70c57828b-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:24:43.830401 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:43.830370    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"
Jul 17 06:24:43.830630 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:43.830368    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9" event={"ID":"1edee576-1c31-4ccf-8af4-ebf70c57828b","Type":"ContainerDied","Data":"70c818c47dc016ce71fcccba57cd82163d11f56c9f3bb092c6ef8d6c4edfbb96"}
Jul 17 06:24:43.830630 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:43.830526    2558 scope.go:117] "RemoveContainer" containerID="e99296150bd0fbe98d168c40ae181c78770c3beef04e6c09e7f51618b34a7127"
Jul 17 06:24:43.852064 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:43.852033    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"]
Jul 17 06:24:43.858574 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:43.858548    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-s2tc9"]
Jul 17 06:24:44.079745 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:24:44.079715    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b" path="/var/lib/kubelet/pods/1edee576-1c31-4ccf-8af4-ebf70c57828b/volumes"
Jul 17 06:25:25.939788 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.939753    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"]
Jul 17 06:25:25.940231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940145    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerName="revoke-keys"
Jul 17 06:25:25.940231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940156    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerName="revoke-keys"
Jul 17 06:25:25.940231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940164    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerName="revoke-keys"
Jul 17 06:25:25.940231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940170    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerName="revoke-keys"
Jul 17 06:25:25.940231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940184    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerName="revoke-keys"
Jul 17 06:25:25.940231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940190    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerName="revoke-keys"
Jul 17 06:25:25.940231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940196    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerName="revoke-keys"
Jul 17 06:25:25.940231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940201    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerName="revoke-keys"
Jul 17 06:25:25.940231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940207    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerName="revoke-keys"
Jul 17 06:25:25.940231 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940212    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerName="revoke-keys"
Jul 17 06:25:25.940580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940283    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerName="revoke-keys"
Jul 17 06:25:25.940580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940293    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerName="revoke-keys"
Jul 17 06:25:25.940580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940299    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerName="revoke-keys"
Jul 17 06:25:25.940580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940307    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae3a9e1c-f840-40d9-bd21-e949e055c29c" containerName="revoke-keys"
Jul 17 06:25:25.940580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.940314    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerName="revoke-keys"
Jul 17 06:25:25.943389 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.943373    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"
Jul 17 06:25:25.946026 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.946002    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:25:25.950990 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:25.950746    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"]
Jul 17 06:25:26.020444 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:26.020417    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ba0ad11-6a41-419d-9169-65f70321b07f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-xfc98\" (UID: \"0ba0ad11-6a41-419d-9169-65f70321b07f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"
Jul 17 06:25:26.121115 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:26.121091    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ba0ad11-6a41-419d-9169-65f70321b07f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-xfc98\" (UID: \"0ba0ad11-6a41-419d-9169-65f70321b07f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"
Jul 17 06:25:26.121656 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:26.121638    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ba0ad11-6a41-419d-9169-65f70321b07f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-xfc98\" (UID: \"0ba0ad11-6a41-419d-9169-65f70321b07f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"
Jul 17 06:25:26.255508 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:26.255437    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"
Jul 17 06:25:26.394498 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:26.394455    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"]
Jul 17 06:25:26.396014 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:25:26.395988    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ba0ad11_6a41_419d_9169_65f70321b07f.slice/crio-5b0610a9ccdd24ca7c63bf276b87ca9ea0de51d399ca06f08b9f32dcb3d36358 WatchSource:0}: Error finding container 5b0610a9ccdd24ca7c63bf276b87ca9ea0de51d399ca06f08b9f32dcb3d36358: Status 404 returned error can't find the container with id 5b0610a9ccdd24ca7c63bf276b87ca9ea0de51d399ca06f08b9f32dcb3d36358
Jul 17 06:25:27.082628 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:27.082602    2558 generic.go:356] "Generic (PLEG): container finished" podID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerID="fd22c92fd5a340b73fffce037d64de9e9e54a11c866e954e616b2a9248495a85" exitCode=6
Jul 17 06:25:27.082945 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:27.082675    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98" event={"ID":"0ba0ad11-6a41-419d-9169-65f70321b07f","Type":"ContainerDied","Data":"fd22c92fd5a340b73fffce037d64de9e9e54a11c866e954e616b2a9248495a85"}
Jul 17 06:25:27.082945 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:27.082712    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98" event={"ID":"0ba0ad11-6a41-419d-9169-65f70321b07f","Type":"ContainerStarted","Data":"5b0610a9ccdd24ca7c63bf276b87ca9ea0de51d399ca06f08b9f32dcb3d36358"}
Jul 17 06:25:27.082945 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:27.082814    2558 scope.go:117] "RemoveContainer" containerID="fd22c92fd5a340b73fffce037d64de9e9e54a11c866e954e616b2a9248495a85"
Jul 17 06:25:28.091187 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:28.091151    2558 generic.go:356] "Generic (PLEG): container finished" podID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerID="59d5143bb8feeed2211463004b36a87513e8e11f56f0849a130dd5a9fd7f84f5" exitCode=6
Jul 17 06:25:28.091731 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:28.091209    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98" event={"ID":"0ba0ad11-6a41-419d-9169-65f70321b07f","Type":"ContainerDied","Data":"59d5143bb8feeed2211463004b36a87513e8e11f56f0849a130dd5a9fd7f84f5"}
Jul 17 06:25:28.091731 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:28.091260    2558 scope.go:117] "RemoveContainer" containerID="fd22c92fd5a340b73fffce037d64de9e9e54a11c866e954e616b2a9248495a85"
Jul 17 06:25:28.091731 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:28.091553    2558 scope.go:117] "RemoveContainer" containerID="59d5143bb8feeed2211463004b36a87513e8e11f56f0849a130dd5a9fd7f84f5"
Jul 17 06:25:28.091892 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:25:28.091847    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-xfc98_odh-ai-gateway-infra(0ba0ad11-6a41-419d-9169-65f70321b07f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f"
Jul 17 06:25:29.099169 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:29.099142    2558 scope.go:117] "RemoveContainer" containerID="59d5143bb8feeed2211463004b36a87513e8e11f56f0849a130dd5a9fd7f84f5"
Jul 17 06:25:29.099597 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:25:29.099386    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-xfc98_odh-ai-gateway-infra(0ba0ad11-6a41-419d-9169-65f70321b07f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f"
Jul 17 06:25:44.072946 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:44.072911    2558 scope.go:117] "RemoveContainer" containerID="59d5143bb8feeed2211463004b36a87513e8e11f56f0849a130dd5a9fd7f84f5"
Jul 17 06:25:45.188373 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:45.188336    2558 generic.go:356] "Generic (PLEG): container finished" podID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerID="09bae3b71da49bb6ba8f7ba793f4463cb1be9b79d980df21d4258e7e3f5e32f6" exitCode=6
Jul 17 06:25:45.188830 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:45.188400    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98" event={"ID":"0ba0ad11-6a41-419d-9169-65f70321b07f","Type":"ContainerDied","Data":"09bae3b71da49bb6ba8f7ba793f4463cb1be9b79d980df21d4258e7e3f5e32f6"}
Jul 17 06:25:45.188830 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:45.188457    2558 scope.go:117] "RemoveContainer" containerID="59d5143bb8feeed2211463004b36a87513e8e11f56f0849a130dd5a9fd7f84f5"
Jul 17 06:25:45.188830 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:45.188810    2558 scope.go:117] "RemoveContainer" containerID="09bae3b71da49bb6ba8f7ba793f4463cb1be9b79d980df21d4258e7e3f5e32f6"
Jul 17 06:25:45.189087 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:25:45.189062    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-xfc98_odh-ai-gateway-infra(0ba0ad11-6a41-419d-9169-65f70321b07f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f"
Jul 17 06:25:46.220865 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:46.220832    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"]
Jul 17 06:25:46.362101 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:46.362073    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"
Jul 17 06:25:46.517704 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:46.517630    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ba0ad11-6a41-419d-9169-65f70321b07f-maas-api-service-ca\") pod \"0ba0ad11-6a41-419d-9169-65f70321b07f\" (UID: \"0ba0ad11-6a41-419d-9169-65f70321b07f\") "
Jul 17 06:25:46.517950 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:46.517930    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ba0ad11-6a41-419d-9169-65f70321b07f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ba0ad11-6a41-419d-9169-65f70321b07f" (UID: "0ba0ad11-6a41-419d-9169-65f70321b07f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:25:46.618196 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:46.618160    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ba0ad11-6a41-419d-9169-65f70321b07f-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:25:47.201266 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:47.201233    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"
Jul 17 06:25:47.201458 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:47.201263    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98" event={"ID":"0ba0ad11-6a41-419d-9169-65f70321b07f","Type":"ContainerDied","Data":"5b0610a9ccdd24ca7c63bf276b87ca9ea0de51d399ca06f08b9f32dcb3d36358"}
Jul 17 06:25:47.201458 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:47.201327    2558 scope.go:117] "RemoveContainer" containerID="09bae3b71da49bb6ba8f7ba793f4463cb1be9b79d980df21d4258e7e3f5e32f6"
Jul 17 06:25:47.224534 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:47.224508    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"]
Jul 17 06:25:47.230554 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:47.230534    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-xfc98"]
Jul 17 06:25:48.083439 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:25:48.083395    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f" path="/var/lib/kubelet/pods/0ba0ad11-6a41-419d-9169-65f70321b07f/volumes"
Jul 17 06:26:09.807436 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:09.807347    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"]
Jul 17 06:26:09.808074 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:09.807623    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerName="istio-proxy" containerID="cri-o://5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9" gracePeriod=30
Jul 17 06:26:15.068910 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.068851    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:26:15.176689 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.176659    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-token\") pod \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") "
Jul 17 06:26:15.176889 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.176718    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zg62\" (UniqueName: \"kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-kube-api-access-7zg62\") pod \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") "
Jul 17 06:26:15.176889 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.176761    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-envoy\") pod \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") "
Jul 17 06:26:15.176889 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.176815    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-credential-socket\") pod \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") "
Jul 17 06:26:15.176889 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.176848    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-certs\") pod \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") "
Jul 17 06:26:15.177113 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.176896    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-podinfo\") pod \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") "
Jul 17 06:26:15.177113 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.176945    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istiod-ca-cert\") pod \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") "
Jul 17 06:26:15.177113 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.176973    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-socket\") pod \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") "
Jul 17 06:26:15.177113 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.177009    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-data\") pod \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\" (UID: \"02a45d1c-f206-4f8e-a24d-42fe3dc471bf\") "
Jul 17 06:26:15.177113 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.177077    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "02a45d1c-f206-4f8e-a24d-42fe3dc471bf" (UID: "02a45d1c-f206-4f8e-a24d-42fe3dc471bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:26:15.177376 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.177356    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:26:15.177587 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.177557    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "02a45d1c-f206-4f8e-a24d-42fe3dc471bf" (UID: "02a45d1c-f206-4f8e-a24d-42fe3dc471bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:26:15.177691 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.177567    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "02a45d1c-f206-4f8e-a24d-42fe3dc471bf" (UID: "02a45d1c-f206-4f8e-a24d-42fe3dc471bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:26:15.177921 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.177898    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "02a45d1c-f206-4f8e-a24d-42fe3dc471bf" (UID: "02a45d1c-f206-4f8e-a24d-42fe3dc471bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:26:15.178879 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.178841    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "02a45d1c-f206-4f8e-a24d-42fe3dc471bf" (UID: "02a45d1c-f206-4f8e-a24d-42fe3dc471bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:26:15.179798 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.179772    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-kube-api-access-7zg62" (OuterVolumeSpecName: "kube-api-access-7zg62") pod "02a45d1c-f206-4f8e-a24d-42fe3dc471bf" (UID: "02a45d1c-f206-4f8e-a24d-42fe3dc471bf"). InnerVolumeSpecName "kube-api-access-7zg62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:26:15.179929 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.179902    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "02a45d1c-f206-4f8e-a24d-42fe3dc471bf" (UID: "02a45d1c-f206-4f8e-a24d-42fe3dc471bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:26:15.180464 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.180433    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "02a45d1c-f206-4f8e-a24d-42fe3dc471bf" (UID: "02a45d1c-f206-4f8e-a24d-42fe3dc471bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:26:15.181042 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.181016    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "02a45d1c-f206-4f8e-a24d-42fe3dc471bf" (UID: "02a45d1c-f206-4f8e-a24d-42fe3dc471bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:26:15.278685 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.278660    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:26:15.278685 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.278685    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:26:15.278824 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.278694    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:26:15.278824 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.278703    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:26:15.278824 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.278711    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:26:15.278824 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.278719    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zg62\" (UniqueName: \"kubernetes.io/projected/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-kube-api-access-7zg62\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:26:15.278824 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.278727    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:26:15.278824 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.278735    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02a45d1c-f206-4f8e-a24d-42fe3dc471bf-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:26:15.356107 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.356074    2558 generic.go:356] "Generic (PLEG): container finished" podID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerID="5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9" exitCode=0
Jul 17 06:26:15.356334 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.356145    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"
Jul 17 06:26:15.356334 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.356156    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" event={"ID":"02a45d1c-f206-4f8e-a24d-42fe3dc471bf","Type":"ContainerDied","Data":"5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9"}
Jul 17 06:26:15.356334 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.356224    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf" event={"ID":"02a45d1c-f206-4f8e-a24d-42fe3dc471bf","Type":"ContainerDied","Data":"489537f15a1815aa64ebf94c80aaeb602ed1b332a77db1ec6eef9fc6235fdf23"}
Jul 17 06:26:15.356334 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.356262    2558 scope.go:117] "RemoveContainer" containerID="5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9"
Jul 17 06:26:15.366189 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.366173    2558 scope.go:117] "RemoveContainer" containerID="eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f"
Jul 17 06:26:15.374053 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.374037    2558 scope.go:117] "RemoveContainer" containerID="5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9"
Jul 17 06:26:15.374298 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:15.374281    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9\": container with ID starting with 5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9 not found: ID does not exist" containerID="5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9"
Jul 17 06:26:15.374342 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.374305    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9"} err="failed to get container status \"5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9\": rpc error: code = NotFound desc = could not find container \"5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9\": container with ID starting with 5a087813c26a658e12b3a69e0cd3e575862936d036aff59f7535e9473df6aad9 not found: ID does not exist"
Jul 17 06:26:15.374342 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.374322    2558 scope.go:117] "RemoveContainer" containerID="eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f"
Jul 17 06:26:15.374558 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:15.374540    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f\": container with ID starting with eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f not found: ID does not exist" containerID="eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f"
Jul 17 06:26:15.374598 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.374565    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f"} err="failed to get container status \"eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f\": rpc error: code = NotFound desc = could not find container \"eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f\": container with ID starting with eca8870bf75f8b4b9c9f673c8976609083aabe0a53e2272af2bd5b65a2cc357f not found: ID does not exist"
Jul 17 06:26:15.379825 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.379802    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"]
Jul 17 06:26:15.385388 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:15.385361    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-51e238b9-openshift-default-7458554b6c-jrpmf"]
Jul 17 06:26:16.078873 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:16.078835    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" path="/var/lib/kubelet/pods/02a45d1c-f206-4f8e-a24d-42fe3dc471bf/volumes"
Jul 17 06:26:18.199528 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:18.199488    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:26:18.202916 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:18.202893    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:26:18.205313 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:18.205292    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:26:18.208703 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:18.208685    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:26:22.283462 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.283432    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"]
Jul 17 06:26:22.283878 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.283862    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerName="istio-proxy"
Jul 17 06:26:22.283923 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.283880    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerName="istio-proxy"
Jul 17 06:26:22.283923 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.283888    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerName="revoke-keys"
Jul 17 06:26:22.283923 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.283894    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerName="revoke-keys"
Jul 17 06:26:22.283923 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.283917    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerName="revoke-keys"
Jul 17 06:26:22.283923 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.283922    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerName="revoke-keys"
Jul 17 06:26:22.284076 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.283930    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerName="revoke-keys"
Jul 17 06:26:22.284076 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.283934    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerName="revoke-keys"
Jul 17 06:26:22.284076 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.283993    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerName="revoke-keys"
Jul 17 06:26:22.284076 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.284001    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1edee576-1c31-4ccf-8af4-ebf70c57828b" containerName="revoke-keys"
Jul 17 06:26:22.284076 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.284007    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerName="revoke-keys"
Jul 17 06:26:22.284076 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.284013    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerName="istio-proxy"
Jul 17 06:26:22.284076 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.284019    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerName="istio-proxy"
Jul 17 06:26:22.288627 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.288610    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"
Jul 17 06:26:22.291823 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.291802    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:26:22.298222 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.298200    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"]
Jul 17 06:26:22.438815 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.438782    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fc58244-581f-496d-9eb7-5073a3d604e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-8prlq\" (UID: \"3fc58244-581f-496d-9eb7-5073a3d604e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"
Jul 17 06:26:22.539382 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.539323    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fc58244-581f-496d-9eb7-5073a3d604e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-8prlq\" (UID: \"3fc58244-581f-496d-9eb7-5073a3d604e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"
Jul 17 06:26:22.539947 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.539928    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fc58244-581f-496d-9eb7-5073a3d604e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-8prlq\" (UID: \"3fc58244-581f-496d-9eb7-5073a3d604e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"
Jul 17 06:26:22.599850 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.599821    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"
Jul 17 06:26:22.744597 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:22.744551    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"]
Jul 17 06:26:22.746978 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:26:22.746952    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc58244_581f_496d_9eb7_5073a3d604e0.slice/crio-79a20eace5ce0807cc343f37b3d22db530008afa3af267b9d75eae8ee9b53767 WatchSource:0}: Error finding container 79a20eace5ce0807cc343f37b3d22db530008afa3af267b9d75eae8ee9b53767: Status 404 returned error can't find the container with id 79a20eace5ce0807cc343f37b3d22db530008afa3af267b9d75eae8ee9b53767
Jul 17 06:26:23.403556 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:23.403519    2558 generic.go:356] "Generic (PLEG): container finished" podID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerID="73d34353682c946fdf8c32b779116fe63ed649d75b38821cca7cbfa7a997c133" exitCode=6
Jul 17 06:26:23.403893 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:23.403604    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq" event={"ID":"3fc58244-581f-496d-9eb7-5073a3d604e0","Type":"ContainerDied","Data":"73d34353682c946fdf8c32b779116fe63ed649d75b38821cca7cbfa7a997c133"}
Jul 17 06:26:23.403893 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:23.403657    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq" event={"ID":"3fc58244-581f-496d-9eb7-5073a3d604e0","Type":"ContainerStarted","Data":"79a20eace5ce0807cc343f37b3d22db530008afa3af267b9d75eae8ee9b53767"}
Jul 17 06:26:23.403893 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:23.403773    2558 scope.go:117] "RemoveContainer" containerID="73d34353682c946fdf8c32b779116fe63ed649d75b38821cca7cbfa7a997c133"
Jul 17 06:26:24.410364 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:24.410290    2558 generic.go:356] "Generic (PLEG): container finished" podID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerID="9102bbba61ad0991b575cdc94b9184330d0afe360d1b1ecdd6c5d326e30d5112" exitCode=6
Jul 17 06:26:24.410771 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:24.410385    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq" event={"ID":"3fc58244-581f-496d-9eb7-5073a3d604e0","Type":"ContainerDied","Data":"9102bbba61ad0991b575cdc94b9184330d0afe360d1b1ecdd6c5d326e30d5112"}
Jul 17 06:26:24.410771 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:24.410460    2558 scope.go:117] "RemoveContainer" containerID="73d34353682c946fdf8c32b779116fe63ed649d75b38821cca7cbfa7a997c133"
Jul 17 06:26:24.410771 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:24.410600    2558 scope.go:117] "RemoveContainer" containerID="9102bbba61ad0991b575cdc94b9184330d0afe360d1b1ecdd6c5d326e30d5112"
Jul 17 06:26:24.410953 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:24.410854    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-8prlq_odh-ai-gateway-infra(3fc58244-581f-496d-9eb7-5073a3d604e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0"
Jul 17 06:26:25.416935 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:25.416904    2558 scope.go:117] "RemoveContainer" containerID="9102bbba61ad0991b575cdc94b9184330d0afe360d1b1ecdd6c5d326e30d5112"
Jul 17 06:26:25.417401 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:25.417115    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-8prlq_odh-ai-gateway-infra(3fc58244-581f-496d-9eb7-5073a3d604e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0"
Jul 17 06:26:26.006178 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.006140    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"]
Jul 17 06:26:26.006648 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.006623    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerName="revoke-keys"
Jul 17 06:26:26.006648 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.006639    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerName="revoke-keys"
Jul 17 06:26:26.006757 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.006662    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerName="istio-proxy"
Jul 17 06:26:26.006757 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.006669    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="02a45d1c-f206-4f8e-a24d-42fe3dc471bf" containerName="istio-proxy"
Jul 17 06:26:26.006757 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.006748    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ba0ad11-6a41-419d-9169-65f70321b07f" containerName="revoke-keys"
Jul 17 06:26:26.010777 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.010749    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"
Jul 17 06:26:26.020941 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.020915    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"]
Jul 17 06:26:26.171059 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.171023    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c039a473-ddb6-409b-8ceb-cf01233b96ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp\" (UID: \"c039a473-ddb6-409b-8ceb-cf01233b96ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"
Jul 17 06:26:26.271819 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.271756    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c039a473-ddb6-409b-8ceb-cf01233b96ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp\" (UID: \"c039a473-ddb6-409b-8ceb-cf01233b96ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"
Jul 17 06:26:26.272319 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.272300    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c039a473-ddb6-409b-8ceb-cf01233b96ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp\" (UID: \"c039a473-ddb6-409b-8ceb-cf01233b96ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"
Jul 17 06:26:26.322643 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.322620    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"
Jul 17 06:26:26.469513 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:26.469489    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"]
Jul 17 06:26:26.471383 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:26:26.471355    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc039a473_ddb6_409b_8ceb_cf01233b96ab.slice/crio-c825df88bd25ab61cad0557e38e704368251ad2cd74a0101c4c6f34e2e27a8ea WatchSource:0}: Error finding container c825df88bd25ab61cad0557e38e704368251ad2cd74a0101c4c6f34e2e27a8ea: Status 404 returned error can't find the container with id c825df88bd25ab61cad0557e38e704368251ad2cd74a0101c4c6f34e2e27a8ea
Jul 17 06:26:27.430265 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:27.430221    2558 generic.go:356] "Generic (PLEG): container finished" podID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerID="faf9a254f3487cefd8d0f870114dff36eda70e6ec79ab102a31e6e041b55b8e9" exitCode=6
Jul 17 06:26:27.430455 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:27.430294    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp" event={"ID":"c039a473-ddb6-409b-8ceb-cf01233b96ab","Type":"ContainerDied","Data":"faf9a254f3487cefd8d0f870114dff36eda70e6ec79ab102a31e6e041b55b8e9"}
Jul 17 06:26:27.430455 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:27.430350    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp" event={"ID":"c039a473-ddb6-409b-8ceb-cf01233b96ab","Type":"ContainerStarted","Data":"c825df88bd25ab61cad0557e38e704368251ad2cd74a0101c4c6f34e2e27a8ea"}
Jul 17 06:26:27.431027 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:27.431003    2558 scope.go:117] "RemoveContainer" containerID="faf9a254f3487cefd8d0f870114dff36eda70e6ec79ab102a31e6e041b55b8e9"
Jul 17 06:26:28.437575 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:28.437463    2558 generic.go:356] "Generic (PLEG): container finished" podID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerID="946f912f4c8775bd9deba4fcc1afcf1a0f64999215fa8e1e6a8be95caa782458" exitCode=6
Jul 17 06:26:28.437575 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:28.437505    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp" event={"ID":"c039a473-ddb6-409b-8ceb-cf01233b96ab","Type":"ContainerDied","Data":"946f912f4c8775bd9deba4fcc1afcf1a0f64999215fa8e1e6a8be95caa782458"}
Jul 17 06:26:28.438041 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:28.437576    2558 scope.go:117] "RemoveContainer" containerID="faf9a254f3487cefd8d0f870114dff36eda70e6ec79ab102a31e6e041b55b8e9"
Jul 17 06:26:28.438041 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:28.437763    2558 scope.go:117] "RemoveContainer" containerID="946f912f4c8775bd9deba4fcc1afcf1a0f64999215fa8e1e6a8be95caa782458"
Jul 17 06:26:28.438041 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:28.437965    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp_odh-ai-gateway-infra(c039a473-ddb6-409b-8ceb-cf01233b96ab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab"
Jul 17 06:26:29.445091 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:29.445063    2558 scope.go:117] "RemoveContainer" containerID="946f912f4c8775bd9deba4fcc1afcf1a0f64999215fa8e1e6a8be95caa782458"
Jul 17 06:26:29.445557 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:29.445353    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp_odh-ai-gateway-infra(c039a473-ddb6-409b-8ceb-cf01233b96ab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab"
Jul 17 06:26:37.073642 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:37.073587    2558 scope.go:117] "RemoveContainer" containerID="9102bbba61ad0991b575cdc94b9184330d0afe360d1b1ecdd6c5d326e30d5112"
Jul 17 06:26:37.489743 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:37.489688    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq" event={"ID":"3fc58244-581f-496d-9eb7-5073a3d604e0","Type":"ContainerStarted","Data":"f9c0dcacee288dc4bb28c35325cb3f0370e9a87de592d4c47e57298c58308c02"}
Jul 17 06:26:37.511234 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:37.511151    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq" podStartSLOduration=15.511128088 podStartE2EDuration="15.511128088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:26:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:26:37.509847933 +0000 UTC m=+2719.951786509" watchObservedRunningTime="2026-07-17 06:26:37.511128088 +0000 UTC m=+2719.953066803"
Jul 17 06:26:38.497681 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:38.497582    2558 generic.go:356] "Generic (PLEG): container finished" podID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerID="f9c0dcacee288dc4bb28c35325cb3f0370e9a87de592d4c47e57298c58308c02" exitCode=6
Jul 17 06:26:38.497681 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:38.497655    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq" event={"ID":"3fc58244-581f-496d-9eb7-5073a3d604e0","Type":"ContainerDied","Data":"f9c0dcacee288dc4bb28c35325cb3f0370e9a87de592d4c47e57298c58308c02"}
Jul 17 06:26:38.498174 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:38.497723    2558 scope.go:117] "RemoveContainer" containerID="9102bbba61ad0991b575cdc94b9184330d0afe360d1b1ecdd6c5d326e30d5112"
Jul 17 06:26:38.498174 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:38.497977    2558 scope.go:117] "RemoveContainer" containerID="f9c0dcacee288dc4bb28c35325cb3f0370e9a87de592d4c47e57298c58308c02"
Jul 17 06:26:38.498258 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:38.498234    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-8prlq_odh-ai-gateway-infra(3fc58244-581f-496d-9eb7-5073a3d604e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0"
Jul 17 06:26:38.528112 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:38.528068    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"]
Jul 17 06:26:39.650450 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:39.650419    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"
Jul 17 06:26:39.796262 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:39.796166    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fc58244-581f-496d-9eb7-5073a3d604e0-maas-api-service-ca\") pod \"3fc58244-581f-496d-9eb7-5073a3d604e0\" (UID: \"3fc58244-581f-496d-9eb7-5073a3d604e0\") "
Jul 17 06:26:39.796559 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:39.796538    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc58244-581f-496d-9eb7-5073a3d604e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3fc58244-581f-496d-9eb7-5073a3d604e0" (UID: "3fc58244-581f-496d-9eb7-5073a3d604e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:26:39.903916 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:39.897501    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fc58244-581f-496d-9eb7-5073a3d604e0-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:26:40.512109 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:40.512015    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq" event={"ID":"3fc58244-581f-496d-9eb7-5073a3d604e0","Type":"ContainerDied","Data":"79a20eace5ce0807cc343f37b3d22db530008afa3af267b9d75eae8ee9b53767"}
Jul 17 06:26:40.512109 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:40.512059    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"
Jul 17 06:26:40.512109 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:40.512087    2558 scope.go:117] "RemoveContainer" containerID="f9c0dcacee288dc4bb28c35325cb3f0370e9a87de592d4c47e57298c58308c02"
Jul 17 06:26:40.529833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:40.529802    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"]
Jul 17 06:26:40.535735 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:40.535709    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-8prlq"]
Jul 17 06:26:42.072705 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:42.072668    2558 scope.go:117] "RemoveContainer" containerID="946f912f4c8775bd9deba4fcc1afcf1a0f64999215fa8e1e6a8be95caa782458"
Jul 17 06:26:42.079218 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:42.079183    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0" path="/var/lib/kubelet/pods/3fc58244-581f-496d-9eb7-5073a3d604e0/volumes"
Jul 17 06:26:42.526897 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:42.526855    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp" event={"ID":"c039a473-ddb6-409b-8ceb-cf01233b96ab","Type":"ContainerStarted","Data":"b46f1685c9d12cf999a5ed25cb0402204980884b07e4b0831077f1bea462bfcb"}
Jul 17 06:26:42.543967 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:42.543911    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp" podStartSLOduration=17.543890569 podStartE2EDuration="17.543890569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:26:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:26:42.5433838 +0000 UTC m=+2724.985322582" watchObservedRunningTime="2026-07-17 06:26:42.543890569 +0000 UTC m=+2724.985829147"
Jul 17 06:26:43.534269 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:43.534234    2558 generic.go:356] "Generic (PLEG): container finished" podID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerID="b46f1685c9d12cf999a5ed25cb0402204980884b07e4b0831077f1bea462bfcb" exitCode=6
Jul 17 06:26:43.534754 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:43.534284    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp" event={"ID":"c039a473-ddb6-409b-8ceb-cf01233b96ab","Type":"ContainerDied","Data":"b46f1685c9d12cf999a5ed25cb0402204980884b07e4b0831077f1bea462bfcb"}
Jul 17 06:26:43.534754 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:43.534332    2558 scope.go:117] "RemoveContainer" containerID="946f912f4c8775bd9deba4fcc1afcf1a0f64999215fa8e1e6a8be95caa782458"
Jul 17 06:26:43.534754 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:43.534631    2558 scope.go:117] "RemoveContainer" containerID="b46f1685c9d12cf999a5ed25cb0402204980884b07e4b0831077f1bea462bfcb"
Jul 17 06:26:43.534878 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:43.534852    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp_odh-ai-gateway-infra(c039a473-ddb6-409b-8ceb-cf01233b96ab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab"
Jul 17 06:26:43.587958 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:43.587926    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"]
Jul 17 06:26:44.684880 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:44.684854    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"
Jul 17 06:26:44.741047 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:44.741013    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c039a473-ddb6-409b-8ceb-cf01233b96ab-maas-api-service-ca\") pod \"c039a473-ddb6-409b-8ceb-cf01233b96ab\" (UID: \"c039a473-ddb6-409b-8ceb-cf01233b96ab\") "
Jul 17 06:26:44.741377 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:44.741352    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c039a473-ddb6-409b-8ceb-cf01233b96ab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c039a473-ddb6-409b-8ceb-cf01233b96ab" (UID: "c039a473-ddb6-409b-8ceb-cf01233b96ab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:26:44.842083 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:44.842049    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c039a473-ddb6-409b-8ceb-cf01233b96ab-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:26:45.548260 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:45.548232    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"
Jul 17 06:26:45.548260 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:45.548241    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp" event={"ID":"c039a473-ddb6-409b-8ceb-cf01233b96ab","Type":"ContainerDied","Data":"c825df88bd25ab61cad0557e38e704368251ad2cd74a0101c4c6f34e2e27a8ea"}
Jul 17 06:26:45.548570 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:45.548313    2558 scope.go:117] "RemoveContainer" containerID="b46f1685c9d12cf999a5ed25cb0402204980884b07e4b0831077f1bea462bfcb"
Jul 17 06:26:45.569866 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:45.569844    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"]
Jul 17 06:26:45.574009 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:45.573991    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tlkdp"]
Jul 17 06:26:46.080082 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:46.080050    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab" path="/var/lib/kubelet/pods/c039a473-ddb6-409b-8ceb-cf01233b96ab/volumes"
Jul 17 06:26:49.889617 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.889587    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"]
Jul 17 06:26:49.889987 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.889979    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerName="revoke-keys"
Jul 17 06:26:49.890031 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.889990    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerName="revoke-keys"
Jul 17 06:26:49.890031 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890000    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerName="revoke-keys"
Jul 17 06:26:49.890031 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890005    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerName="revoke-keys"
Jul 17 06:26:49.890031 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890012    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerName="revoke-keys"
Jul 17 06:26:49.890031 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890017    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerName="revoke-keys"
Jul 17 06:26:49.890183 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890036    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerName="revoke-keys"
Jul 17 06:26:49.890183 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890041    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerName="revoke-keys"
Jul 17 06:26:49.890183 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890098    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerName="revoke-keys"
Jul 17 06:26:49.890183 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890104    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerName="revoke-keys"
Jul 17 06:26:49.890183 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890110    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerName="revoke-keys"
Jul 17 06:26:49.890183 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890117    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerName="revoke-keys"
Jul 17 06:26:49.890183 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890125    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerName="revoke-keys"
Jul 17 06:26:49.890183 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.890131    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerName="revoke-keys"
Jul 17 06:26:49.894516 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.894498    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"
Jul 17 06:26:49.897168 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.897144    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:26:49.901684 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.901661    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"]
Jul 17 06:26:49.988978 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:49.988957    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab0200f-0d34-45ce-a25c-dd1673061620-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw\" (UID: \"9ab0200f-0d34-45ce-a25c-dd1673061620\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"
Jul 17 06:26:50.089822 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:50.089793    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab0200f-0d34-45ce-a25c-dd1673061620-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw\" (UID: \"9ab0200f-0d34-45ce-a25c-dd1673061620\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"
Jul 17 06:26:50.090414 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:50.090388    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab0200f-0d34-45ce-a25c-dd1673061620-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw\" (UID: \"9ab0200f-0d34-45ce-a25c-dd1673061620\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"
Jul 17 06:26:50.206677 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:50.206620    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"
Jul 17 06:26:50.555366 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:50.555342    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"]
Jul 17 06:26:50.557915 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:26:50.557886    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab0200f_0d34_45ce_a25c_dd1673061620.slice/crio-8f0c7f47ba8969e281585b843df7fd8f27701073438e6c915c50db5197b82f9f WatchSource:0}: Error finding container 8f0c7f47ba8969e281585b843df7fd8f27701073438e6c915c50db5197b82f9f: Status 404 returned error can't find the container with id 8f0c7f47ba8969e281585b843df7fd8f27701073438e6c915c50db5197b82f9f
Jul 17 06:26:50.578805 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:50.578780    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw" event={"ID":"9ab0200f-0d34-45ce-a25c-dd1673061620","Type":"ContainerStarted","Data":"8f0c7f47ba8969e281585b843df7fd8f27701073438e6c915c50db5197b82f9f"}
Jul 17 06:26:51.593411 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:51.593376    2558 generic.go:356] "Generic (PLEG): container finished" podID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerID="1a6fc1116519e11e8236e0ea999924e85713514f420727602791f7e148d2d25f" exitCode=6
Jul 17 06:26:51.593832 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:51.593426    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw" event={"ID":"9ab0200f-0d34-45ce-a25c-dd1673061620","Type":"ContainerDied","Data":"1a6fc1116519e11e8236e0ea999924e85713514f420727602791f7e148d2d25f"}
Jul 17 06:26:51.593832 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:51.593702    2558 scope.go:117] "RemoveContainer" containerID="1a6fc1116519e11e8236e0ea999924e85713514f420727602791f7e148d2d25f"
Jul 17 06:26:52.602117 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:52.602084    2558 generic.go:356] "Generic (PLEG): container finished" podID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerID="7da9e6586ee698db58aab678664ea6c3e4fb54eaf7be1a83438f2e3d16290623" exitCode=6
Jul 17 06:26:52.602486 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:52.602141    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw" event={"ID":"9ab0200f-0d34-45ce-a25c-dd1673061620","Type":"ContainerDied","Data":"7da9e6586ee698db58aab678664ea6c3e4fb54eaf7be1a83438f2e3d16290623"}
Jul 17 06:26:52.602486 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:52.602193    2558 scope.go:117] "RemoveContainer" containerID="1a6fc1116519e11e8236e0ea999924e85713514f420727602791f7e148d2d25f"
Jul 17 06:26:52.602486 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:52.602347    2558 scope.go:117] "RemoveContainer" containerID="7da9e6586ee698db58aab678664ea6c3e4fb54eaf7be1a83438f2e3d16290623"
Jul 17 06:26:52.602641 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:52.602576    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw_odh-ai-gateway-infra(9ab0200f-0d34-45ce-a25c-dd1673061620)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620"
Jul 17 06:26:53.609253 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:53.609222    2558 scope.go:117] "RemoveContainer" containerID="7da9e6586ee698db58aab678664ea6c3e4fb54eaf7be1a83438f2e3d16290623"
Jul 17 06:26:53.609799 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:53.609544    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw_odh-ai-gateway-infra(9ab0200f-0d34-45ce-a25c-dd1673061620)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620"
Jul 17 06:26:53.908411 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:53.908325    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"]
Jul 17 06:26:53.908800 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:53.908786    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerName="revoke-keys"
Jul 17 06:26:53.908847 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:53.908802    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc58244-581f-496d-9eb7-5073a3d604e0" containerName="revoke-keys"
Jul 17 06:26:53.908847 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:53.908812    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerName="revoke-keys"
Jul 17 06:26:53.908847 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:53.908817    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c039a473-ddb6-409b-8ceb-cf01233b96ab" containerName="revoke-keys"
Jul 17 06:26:53.913096 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:53.913075    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"
Jul 17 06:26:53.920546 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:53.920511    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"]
Jul 17 06:26:54.029379 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:54.029329    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a24784af-f7d8-49a4-afe7-c5273ee9eff4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf\" (UID: \"a24784af-f7d8-49a4-afe7-c5273ee9eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"
Jul 17 06:26:54.130134 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:54.130094    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a24784af-f7d8-49a4-afe7-c5273ee9eff4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf\" (UID: \"a24784af-f7d8-49a4-afe7-c5273ee9eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"
Jul 17 06:26:54.130757 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:54.130736    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a24784af-f7d8-49a4-afe7-c5273ee9eff4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf\" (UID: \"a24784af-f7d8-49a4-afe7-c5273ee9eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"
Jul 17 06:26:54.226103 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:54.226020    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"
Jul 17 06:26:54.365115 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:54.365093    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"]
Jul 17 06:26:54.367232 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:26:54.367200    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda24784af_f7d8_49a4_afe7_c5273ee9eff4.slice/crio-792fa113357fab3c231185a3b4ed3313e127a0f2a3abbbb0f671c207efae4be3 WatchSource:0}: Error finding container 792fa113357fab3c231185a3b4ed3313e127a0f2a3abbbb0f671c207efae4be3: Status 404 returned error can't find the container with id 792fa113357fab3c231185a3b4ed3313e127a0f2a3abbbb0f671c207efae4be3
Jul 17 06:26:54.615764 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:54.615736    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf" event={"ID":"a24784af-f7d8-49a4-afe7-c5273ee9eff4","Type":"ContainerStarted","Data":"f6973995e87b018bcd65fe21123b07e10fe728d4be9b24514cb0f3054bdaeed2"}
Jul 17 06:26:54.616055 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:54.615785    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf" event={"ID":"a24784af-f7d8-49a4-afe7-c5273ee9eff4","Type":"ContainerStarted","Data":"792fa113357fab3c231185a3b4ed3313e127a0f2a3abbbb0f671c207efae4be3"}
Jul 17 06:26:54.632314 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:54.632273    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf" podStartSLOduration=1.632258913 podStartE2EDuration="1.632258913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:26:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:26:54.6312029 +0000 UTC m=+2737.073141500" watchObservedRunningTime="2026-07-17 06:26:54.632258913 +0000 UTC m=+2737.074197521"
Jul 17 06:26:55.623419 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:55.623378    2558 generic.go:356] "Generic (PLEG): container finished" podID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerID="f6973995e87b018bcd65fe21123b07e10fe728d4be9b24514cb0f3054bdaeed2" exitCode=6
Jul 17 06:26:55.623831 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:55.623455    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf" event={"ID":"a24784af-f7d8-49a4-afe7-c5273ee9eff4","Type":"ContainerDied","Data":"f6973995e87b018bcd65fe21123b07e10fe728d4be9b24514cb0f3054bdaeed2"}
Jul 17 06:26:55.623831 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:55.623670    2558 scope.go:117] "RemoveContainer" containerID="f6973995e87b018bcd65fe21123b07e10fe728d4be9b24514cb0f3054bdaeed2"
Jul 17 06:26:56.630562 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:56.630521    2558 generic.go:356] "Generic (PLEG): container finished" podID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerID="fd0f882d3791f4443245be6a818de18c4adcc877e21804cb2e616dc4251c4341" exitCode=6
Jul 17 06:26:56.631054 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:56.630607    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf" event={"ID":"a24784af-f7d8-49a4-afe7-c5273ee9eff4","Type":"ContainerDied","Data":"fd0f882d3791f4443245be6a818de18c4adcc877e21804cb2e616dc4251c4341"}
Jul 17 06:26:56.631054 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:56.630670    2558 scope.go:117] "RemoveContainer" containerID="f6973995e87b018bcd65fe21123b07e10fe728d4be9b24514cb0f3054bdaeed2"
Jul 17 06:26:56.631054 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:56.630877    2558 scope.go:117] "RemoveContainer" containerID="fd0f882d3791f4443245be6a818de18c4adcc877e21804cb2e616dc4251c4341"
Jul 17 06:26:56.631177 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:56.631139    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf_odh-ai-gateway-infra(a24784af-f7d8-49a4-afe7-c5273ee9eff4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4"
Jul 17 06:26:57.638057 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:26:57.638030    2558 scope.go:117] "RemoveContainer" containerID="fd0f882d3791f4443245be6a818de18c4adcc877e21804cb2e616dc4251c4341"
Jul 17 06:26:57.638418 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:26:57.638218    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf_odh-ai-gateway-infra(a24784af-f7d8-49a4-afe7-c5273ee9eff4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4"
Jul 17 06:27:04.072889 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:04.072857    2558 scope.go:117] "RemoveContainer" containerID="7da9e6586ee698db58aab678664ea6c3e4fb54eaf7be1a83438f2e3d16290623"
Jul 17 06:27:04.678417 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:04.678325    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw" event={"ID":"9ab0200f-0d34-45ce-a25c-dd1673061620","Type":"ContainerStarted","Data":"29fa61a4f4aef7d08462765b5617c40c204b21d5f027e3d87c22d2a5698e39af"}
Jul 17 06:27:04.697235 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:04.697176    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw" podStartSLOduration=15.697159737 podStartE2EDuration="15.697159737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:26:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:27:04.69545452 +0000 UTC m=+2747.137393095" watchObservedRunningTime="2026-07-17 06:27:04.697159737 +0000 UTC m=+2747.139098312"
Jul 17 06:27:05.687594 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:05.687561    2558 generic.go:356] "Generic (PLEG): container finished" podID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerID="29fa61a4f4aef7d08462765b5617c40c204b21d5f027e3d87c22d2a5698e39af" exitCode=6
Jul 17 06:27:05.688051 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:05.687619    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw" event={"ID":"9ab0200f-0d34-45ce-a25c-dd1673061620","Type":"ContainerDied","Data":"29fa61a4f4aef7d08462765b5617c40c204b21d5f027e3d87c22d2a5698e39af"}
Jul 17 06:27:05.688051 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:05.687717    2558 scope.go:117] "RemoveContainer" containerID="7da9e6586ee698db58aab678664ea6c3e4fb54eaf7be1a83438f2e3d16290623"
Jul 17 06:27:05.688051 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:05.687984    2558 scope.go:117] "RemoveContainer" containerID="29fa61a4f4aef7d08462765b5617c40c204b21d5f027e3d87c22d2a5698e39af"
Jul 17 06:27:05.688290 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:05.688267    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw_odh-ai-gateway-infra(9ab0200f-0d34-45ce-a25c-dd1673061620)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620"
Jul 17 06:27:05.729680 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:05.729642    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"]
Jul 17 06:27:06.835559 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:06.835528    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"
Jul 17 06:27:06.945770 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:06.945695    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab0200f-0d34-45ce-a25c-dd1673061620-maas-api-service-ca\") pod \"9ab0200f-0d34-45ce-a25c-dd1673061620\" (UID: \"9ab0200f-0d34-45ce-a25c-dd1673061620\") "
Jul 17 06:27:06.946022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:06.945999    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ab0200f-0d34-45ce-a25c-dd1673061620-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ab0200f-0d34-45ce-a25c-dd1673061620" (UID: "9ab0200f-0d34-45ce-a25c-dd1673061620"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:27:07.046791 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:07.046764    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab0200f-0d34-45ce-a25c-dd1673061620-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:27:07.701426 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:07.701393    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"
Jul 17 06:27:07.701426 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:07.701405    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw" event={"ID":"9ab0200f-0d34-45ce-a25c-dd1673061620","Type":"ContainerDied","Data":"8f0c7f47ba8969e281585b843df7fd8f27701073438e6c915c50db5197b82f9f"}
Jul 17 06:27:07.701681 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:07.701497    2558 scope.go:117] "RemoveContainer" containerID="29fa61a4f4aef7d08462765b5617c40c204b21d5f027e3d87c22d2a5698e39af"
Jul 17 06:27:07.730113 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:07.730082    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"]
Jul 17 06:27:07.734072 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:07.734039    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-kbnbw"]
Jul 17 06:27:08.078278 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:08.078195    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620" path="/var/lib/kubelet/pods/9ab0200f-0d34-45ce-a25c-dd1673061620/volumes"
Jul 17 06:27:10.078881 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:10.078849    2558 scope.go:117] "RemoveContainer" containerID="fd0f882d3791f4443245be6a818de18c4adcc877e21804cb2e616dc4251c4341"
Jul 17 06:27:10.720431 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:10.720397    2558 generic.go:356] "Generic (PLEG): container finished" podID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerID="8851e872db29cfdf31cc1ab2f04c48f0788a29b7b72115561e0edf4358c161df" exitCode=6
Jul 17 06:27:10.720605 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:10.720501    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf" event={"ID":"a24784af-f7d8-49a4-afe7-c5273ee9eff4","Type":"ContainerDied","Data":"8851e872db29cfdf31cc1ab2f04c48f0788a29b7b72115561e0edf4358c161df"}
Jul 17 06:27:10.720605 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:10.720582    2558 scope.go:117] "RemoveContainer" containerID="fd0f882d3791f4443245be6a818de18c4adcc877e21804cb2e616dc4251c4341"
Jul 17 06:27:10.720923 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:10.720894    2558 scope.go:117] "RemoveContainer" containerID="8851e872db29cfdf31cc1ab2f04c48f0788a29b7b72115561e0edf4358c161df"
Jul 17 06:27:10.721134 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:10.721115    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf_odh-ai-gateway-infra(a24784af-f7d8-49a4-afe7-c5273ee9eff4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4"
Jul 17 06:27:11.747901 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:11.747869    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"]
Jul 17 06:27:11.886728 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:11.886705    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"
Jul 17 06:27:11.987927 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:11.987897    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a24784af-f7d8-49a4-afe7-c5273ee9eff4-maas-api-service-ca\") pod \"a24784af-f7d8-49a4-afe7-c5273ee9eff4\" (UID: \"a24784af-f7d8-49a4-afe7-c5273ee9eff4\") "
Jul 17 06:27:11.988212 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:11.988193    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a24784af-f7d8-49a4-afe7-c5273ee9eff4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a24784af-f7d8-49a4-afe7-c5273ee9eff4" (UID: "a24784af-f7d8-49a4-afe7-c5273ee9eff4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:27:12.088810 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.088788    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a24784af-f7d8-49a4-afe7-c5273ee9eff4-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:27:12.347216 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347143    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"]
Jul 17 06:27:12.347598 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347568    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerName="revoke-keys"
Jul 17 06:27:12.347651 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347600    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerName="revoke-keys"
Jul 17 06:27:12.347651 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347610    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerName="revoke-keys"
Jul 17 06:27:12.347651 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347615    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerName="revoke-keys"
Jul 17 06:27:12.347651 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347625    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerName="revoke-keys"
Jul 17 06:27:12.347651 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347630    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerName="revoke-keys"
Jul 17 06:27:12.347651 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347640    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerName="revoke-keys"
Jul 17 06:27:12.347651 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347646    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerName="revoke-keys"
Jul 17 06:27:12.347875 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347657    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerName="revoke-keys"
Jul 17 06:27:12.347875 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347663    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerName="revoke-keys"
Jul 17 06:27:12.347875 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347732    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerName="revoke-keys"
Jul 17 06:27:12.347875 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347739    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerName="revoke-keys"
Jul 17 06:27:12.347875 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347748    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerName="revoke-keys"
Jul 17 06:27:12.347875 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347753    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerName="revoke-keys"
Jul 17 06:27:12.347875 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.347758    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerName="revoke-keys"
Jul 17 06:27:12.350966 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.350948    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"
Jul 17 06:27:12.379170 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.379135    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"]
Jul 17 06:27:12.492254 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.492221    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e1a23b-646f-436c-bf46-c95b9fd913f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-lx7xd\" (UID: \"f5e1a23b-646f-436c-bf46-c95b9fd913f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"
Jul 17 06:27:12.593689 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.593660    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e1a23b-646f-436c-bf46-c95b9fd913f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-lx7xd\" (UID: \"f5e1a23b-646f-436c-bf46-c95b9fd913f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"
Jul 17 06:27:12.594281 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.594258    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e1a23b-646f-436c-bf46-c95b9fd913f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-lx7xd\" (UID: \"f5e1a23b-646f-436c-bf46-c95b9fd913f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"
Jul 17 06:27:12.661196 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.661134    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"
Jul 17 06:27:12.757009 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.756985    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"
Jul 17 06:27:12.757460 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.757013    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf" event={"ID":"a24784af-f7d8-49a4-afe7-c5273ee9eff4","Type":"ContainerDied","Data":"792fa113357fab3c231185a3b4ed3313e127a0f2a3abbbb0f671c207efae4be3"}
Jul 17 06:27:12.757460 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.757084    2558 scope.go:117] "RemoveContainer" containerID="8851e872db29cfdf31cc1ab2f04c48f0788a29b7b72115561e0edf4358c161df"
Jul 17 06:27:12.781952 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.781925    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"]
Jul 17 06:27:12.784829 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:12.784805    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-5vsgf"]
Jul 17 06:27:13.020509 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:13.020492    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"]
Jul 17 06:27:13.022907 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:27:13.022880    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5e1a23b_646f_436c_bf46_c95b9fd913f9.slice/crio-ce5a75cef77bf46da87281311ec99c87d5101c58f0c9997cec82fa10ff2118a1 WatchSource:0}: Error finding container ce5a75cef77bf46da87281311ec99c87d5101c58f0c9997cec82fa10ff2118a1: Status 404 returned error can't find the container with id ce5a75cef77bf46da87281311ec99c87d5101c58f0c9997cec82fa10ff2118a1
Jul 17 06:27:13.765175 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:13.765141    2558 generic.go:356] "Generic (PLEG): container finished" podID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerID="17525938217ecc8a8d53ddd8c92c646041701a35c37f6af0400c89fdd4278126" exitCode=6
Jul 17 06:27:13.765526 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:13.765178    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd" event={"ID":"f5e1a23b-646f-436c-bf46-c95b9fd913f9","Type":"ContainerDied","Data":"17525938217ecc8a8d53ddd8c92c646041701a35c37f6af0400c89fdd4278126"}
Jul 17 06:27:13.765526 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:13.765229    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd" event={"ID":"f5e1a23b-646f-436c-bf46-c95b9fd913f9","Type":"ContainerStarted","Data":"ce5a75cef77bf46da87281311ec99c87d5101c58f0c9997cec82fa10ff2118a1"}
Jul 17 06:27:13.765526 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:13.765374    2558 scope.go:117] "RemoveContainer" containerID="17525938217ecc8a8d53ddd8c92c646041701a35c37f6af0400c89fdd4278126"
Jul 17 06:27:14.078624 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:14.078586    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" path="/var/lib/kubelet/pods/a24784af-f7d8-49a4-afe7-c5273ee9eff4/volumes"
Jul 17 06:27:14.773637 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:14.773596    2558 generic.go:356] "Generic (PLEG): container finished" podID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerID="192b507c7c3a653ac22c565314345aed03a74992d5f8fade444dbf9622e9a36b" exitCode=6
Jul 17 06:27:14.774054 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:14.773689    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd" event={"ID":"f5e1a23b-646f-436c-bf46-c95b9fd913f9","Type":"ContainerDied","Data":"192b507c7c3a653ac22c565314345aed03a74992d5f8fade444dbf9622e9a36b"}
Jul 17 06:27:14.774054 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:14.773755    2558 scope.go:117] "RemoveContainer" containerID="17525938217ecc8a8d53ddd8c92c646041701a35c37f6af0400c89fdd4278126"
Jul 17 06:27:14.774054 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:14.773883    2558 scope.go:117] "RemoveContainer" containerID="192b507c7c3a653ac22c565314345aed03a74992d5f8fade444dbf9622e9a36b"
Jul 17 06:27:14.774186 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:14.774095    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-lx7xd_odh-ai-gateway-infra(f5e1a23b-646f-436c-bf46-c95b9fd913f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9"
Jul 17 06:27:15.782494 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:15.782441    2558 scope.go:117] "RemoveContainer" containerID="192b507c7c3a653ac22c565314345aed03a74992d5f8fade444dbf9622e9a36b"
Jul 17 06:27:15.782879 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:15.782725    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-lx7xd_odh-ai-gateway-infra(f5e1a23b-646f-436c-bf46-c95b9fd913f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9"
Jul 17 06:27:16.040657 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.040572    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"]
Jul 17 06:27:16.041066 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.041047    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerName="revoke-keys"
Jul 17 06:27:16.041161 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.041069    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24784af-f7d8-49a4-afe7-c5273ee9eff4" containerName="revoke-keys"
Jul 17 06:27:16.041218 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.041173    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab0200f-0d34-45ce-a25c-dd1673061620" containerName="revoke-keys"
Jul 17 06:27:16.045389 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.045369    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"
Jul 17 06:27:16.051598 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.051572    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"]
Jul 17 06:27:16.125726 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.125700    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/274c2204-548b-4679-9dcc-d0842260bc46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5\" (UID: \"274c2204-548b-4679-9dcc-d0842260bc46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"
Jul 17 06:27:16.227165 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.227120    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/274c2204-548b-4679-9dcc-d0842260bc46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5\" (UID: \"274c2204-548b-4679-9dcc-d0842260bc46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"
Jul 17 06:27:16.227812 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.227789    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/274c2204-548b-4679-9dcc-d0842260bc46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5\" (UID: \"274c2204-548b-4679-9dcc-d0842260bc46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"
Jul 17 06:27:16.358222 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.358180    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"
Jul 17 06:27:16.505529 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.505461    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"]
Jul 17 06:27:16.508186 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:27:16.508154    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod274c2204_548b_4679_9dcc_d0842260bc46.slice/crio-4c9e9e1863cb7f6582598ce9f8d9f8cf7c46c48d15a8e7863e092b8904a3c00a WatchSource:0}: Error finding container 4c9e9e1863cb7f6582598ce9f8d9f8cf7c46c48d15a8e7863e092b8904a3c00a: Status 404 returned error can't find the container with id 4c9e9e1863cb7f6582598ce9f8d9f8cf7c46c48d15a8e7863e092b8904a3c00a
Jul 17 06:27:16.789014 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:16.788980    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5" event={"ID":"274c2204-548b-4679-9dcc-d0842260bc46","Type":"ContainerStarted","Data":"4c9e9e1863cb7f6582598ce9f8d9f8cf7c46c48d15a8e7863e092b8904a3c00a"}
Jul 17 06:27:17.796425 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:17.796386    2558 generic.go:356] "Generic (PLEG): container finished" podID="274c2204-548b-4679-9dcc-d0842260bc46" containerID="9d479957426bcbc026407641d1ef0a33de647492a3d25a412a0fa1407202a096" exitCode=6
Jul 17 06:27:17.796851 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:17.796434    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5" event={"ID":"274c2204-548b-4679-9dcc-d0842260bc46","Type":"ContainerDied","Data":"9d479957426bcbc026407641d1ef0a33de647492a3d25a412a0fa1407202a096"}
Jul 17 06:27:17.796851 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:17.796762    2558 scope.go:117] "RemoveContainer" containerID="9d479957426bcbc026407641d1ef0a33de647492a3d25a412a0fa1407202a096"
Jul 17 06:27:18.803119 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:18.803019    2558 generic.go:356] "Generic (PLEG): container finished" podID="274c2204-548b-4679-9dcc-d0842260bc46" containerID="e70bb17b9bd7c13d9d1f2ee011feb7bf287ffa4e11ac5b2732a67adda8172ce8" exitCode=6
Jul 17 06:27:18.803119 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:18.803106    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5" event={"ID":"274c2204-548b-4679-9dcc-d0842260bc46","Type":"ContainerDied","Data":"e70bb17b9bd7c13d9d1f2ee011feb7bf287ffa4e11ac5b2732a67adda8172ce8"}
Jul 17 06:27:18.803682 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:18.803165    2558 scope.go:117] "RemoveContainer" containerID="9d479957426bcbc026407641d1ef0a33de647492a3d25a412a0fa1407202a096"
Jul 17 06:27:18.803682 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:18.803388    2558 scope.go:117] "RemoveContainer" containerID="e70bb17b9bd7c13d9d1f2ee011feb7bf287ffa4e11ac5b2732a67adda8172ce8"
Jul 17 06:27:18.803791 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:18.803691    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5_odh-ai-gateway-infra(274c2204-548b-4679-9dcc-d0842260bc46)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5" podUID="274c2204-548b-4679-9dcc-d0842260bc46"
Jul 17 06:27:19.810809 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:19.810779    2558 scope.go:117] "RemoveContainer" containerID="e70bb17b9bd7c13d9d1f2ee011feb7bf287ffa4e11ac5b2732a67adda8172ce8"
Jul 17 06:27:19.811183 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:19.810952    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5_odh-ai-gateway-infra(274c2204-548b-4679-9dcc-d0842260bc46)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5" podUID="274c2204-548b-4679-9dcc-d0842260bc46"
Jul 17 06:27:31.072986 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:31.072954    2558 scope.go:117] "RemoveContainer" containerID="192b507c7c3a653ac22c565314345aed03a74992d5f8fade444dbf9622e9a36b"
Jul 17 06:27:31.881056 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:31.880968    2558 generic.go:356] "Generic (PLEG): container finished" podID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerID="7903f3f2b712dc2503d66c8f85022dcc360ec6c9362de60cd0f47baddf3919d9" exitCode=6
Jul 17 06:27:31.881056 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:31.881039    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd" event={"ID":"f5e1a23b-646f-436c-bf46-c95b9fd913f9","Type":"ContainerDied","Data":"7903f3f2b712dc2503d66c8f85022dcc360ec6c9362de60cd0f47baddf3919d9"}
Jul 17 06:27:31.881306 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:31.881101    2558 scope.go:117] "RemoveContainer" containerID="192b507c7c3a653ac22c565314345aed03a74992d5f8fade444dbf9622e9a36b"
Jul 17 06:27:31.881529 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:31.881510    2558 scope.go:117] "RemoveContainer" containerID="7903f3f2b712dc2503d66c8f85022dcc360ec6c9362de60cd0f47baddf3919d9"
Jul 17 06:27:31.881751 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:31.881725    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-lx7xd_odh-ai-gateway-infra(f5e1a23b-646f-436c-bf46-c95b9fd913f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9"
Jul 17 06:27:32.911723 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:32.911694    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"]
Jul 17 06:27:33.051137 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:33.051112    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"
Jul 17 06:27:33.192153 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:33.192051    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e1a23b-646f-436c-bf46-c95b9fd913f9-maas-api-service-ca\") pod \"f5e1a23b-646f-436c-bf46-c95b9fd913f9\" (UID: \"f5e1a23b-646f-436c-bf46-c95b9fd913f9\") "
Jul 17 06:27:33.192407 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:33.192387    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5e1a23b-646f-436c-bf46-c95b9fd913f9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f5e1a23b-646f-436c-bf46-c95b9fd913f9" (UID: "f5e1a23b-646f-436c-bf46-c95b9fd913f9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:27:33.292863 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:33.292828    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e1a23b-646f-436c-bf46-c95b9fd913f9-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:27:33.896127 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:33.896087    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"
Jul 17 06:27:33.896334 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:33.896125    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd" event={"ID":"f5e1a23b-646f-436c-bf46-c95b9fd913f9","Type":"ContainerDied","Data":"ce5a75cef77bf46da87281311ec99c87d5101c58f0c9997cec82fa10ff2118a1"}
Jul 17 06:27:33.896334 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:33.896195    2558 scope.go:117] "RemoveContainer" containerID="7903f3f2b712dc2503d66c8f85022dcc360ec6c9362de60cd0f47baddf3919d9"
Jul 17 06:27:33.918576 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:33.918536    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"]
Jul 17 06:27:33.924863 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:33.924832    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-lx7xd"]
Jul 17 06:27:34.078239 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:34.078200    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" path="/var/lib/kubelet/pods/f5e1a23b-646f-436c-bf46-c95b9fd913f9/volumes"
Jul 17 06:27:35.072692 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:35.072659    2558 scope.go:117] "RemoveContainer" containerID="e70bb17b9bd7c13d9d1f2ee011feb7bf287ffa4e11ac5b2732a67adda8172ce8"
Jul 17 06:27:35.909170 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:35.909136    2558 generic.go:356] "Generic (PLEG): container finished" podID="274c2204-548b-4679-9dcc-d0842260bc46" containerID="8d9862c6a105b5b27f0bcb08f19243a49a1230d64c21cfe475ab2b3f19d05017" exitCode=6
Jul 17 06:27:35.909352 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:35.909208    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5" event={"ID":"274c2204-548b-4679-9dcc-d0842260bc46","Type":"ContainerDied","Data":"8d9862c6a105b5b27f0bcb08f19243a49a1230d64c21cfe475ab2b3f19d05017"}
Jul 17 06:27:35.909352 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:35.909270    2558 scope.go:117] "RemoveContainer" containerID="e70bb17b9bd7c13d9d1f2ee011feb7bf287ffa4e11ac5b2732a67adda8172ce8"
Jul 17 06:27:35.909580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:35.909564    2558 scope.go:117] "RemoveContainer" containerID="8d9862c6a105b5b27f0bcb08f19243a49a1230d64c21cfe475ab2b3f19d05017"
Jul 17 06:27:35.909858 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:35.909837    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5_odh-ai-gateway-infra(274c2204-548b-4679-9dcc-d0842260bc46)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5" podUID="274c2204-548b-4679-9dcc-d0842260bc46"
Jul 17 06:27:36.940575 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:36.940532    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"]
Jul 17 06:27:37.083680 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:37.083653    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"
Jul 17 06:27:37.124597 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:37.124560    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/274c2204-548b-4679-9dcc-d0842260bc46-maas-api-service-ca\") pod \"274c2204-548b-4679-9dcc-d0842260bc46\" (UID: \"274c2204-548b-4679-9dcc-d0842260bc46\") "
Jul 17 06:27:37.124944 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:37.124915    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/274c2204-548b-4679-9dcc-d0842260bc46-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "274c2204-548b-4679-9dcc-d0842260bc46" (UID: "274c2204-548b-4679-9dcc-d0842260bc46"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:27:37.225538 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:37.225434    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/274c2204-548b-4679-9dcc-d0842260bc46-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:27:37.922239 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:37.922209    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"
Jul 17 06:27:37.922448 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:37.922237    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5" event={"ID":"274c2204-548b-4679-9dcc-d0842260bc46","Type":"ContainerDied","Data":"4c9e9e1863cb7f6582598ce9f8d9f8cf7c46c48d15a8e7863e092b8904a3c00a"}
Jul 17 06:27:37.922448 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:37.922294    2558 scope.go:117] "RemoveContainer" containerID="8d9862c6a105b5b27f0bcb08f19243a49a1230d64c21cfe475ab2b3f19d05017"
Jul 17 06:27:37.944391 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:37.944365    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"]
Jul 17 06:27:37.954304 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:37.954274    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-bm9h5"]
Jul 17 06:27:38.078740 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:38.078708    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="274c2204-548b-4679-9dcc-d0842260bc46" path="/var/lib/kubelet/pods/274c2204-548b-4679-9dcc-d0842260bc46/volumes"
Jul 17 06:27:39.933731 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.933685    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"]
Jul 17 06:27:39.934199 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934139    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="274c2204-548b-4679-9dcc-d0842260bc46" containerName="revoke-keys"
Jul 17 06:27:39.934199 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934152    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="274c2204-548b-4679-9dcc-d0842260bc46" containerName="revoke-keys"
Jul 17 06:27:39.934199 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934162    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerName="revoke-keys"
Jul 17 06:27:39.934199 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934168    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerName="revoke-keys"
Jul 17 06:27:39.934199 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934173    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="274c2204-548b-4679-9dcc-d0842260bc46" containerName="revoke-keys"
Jul 17 06:27:39.934199 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934179    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="274c2204-548b-4679-9dcc-d0842260bc46" containerName="revoke-keys"
Jul 17 06:27:39.934199 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934187    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerName="revoke-keys"
Jul 17 06:27:39.934199 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934192    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerName="revoke-keys"
Jul 17 06:27:39.934199 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934197    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="274c2204-548b-4679-9dcc-d0842260bc46" containerName="revoke-keys"
Jul 17 06:27:39.934199 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934202    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="274c2204-548b-4679-9dcc-d0842260bc46" containerName="revoke-keys"
Jul 17 06:27:39.934662 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934267    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="274c2204-548b-4679-9dcc-d0842260bc46" containerName="revoke-keys"
Jul 17 06:27:39.934662 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934274    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="274c2204-548b-4679-9dcc-d0842260bc46" containerName="revoke-keys"
Jul 17 06:27:39.934662 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934280    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerName="revoke-keys"
Jul 17 06:27:39.934662 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934287    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerName="revoke-keys"
Jul 17 06:27:39.934662 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.934293    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerName="revoke-keys"
Jul 17 06:27:39.938451 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.938431    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"
Jul 17 06:27:39.941256 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.941234    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:27:39.945397 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.945373    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"]
Jul 17 06:27:39.948622 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:39.948257    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-69mc9\" (UID: \"a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"
Jul 17 06:27:40.059555 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:40.059513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-69mc9\" (UID: \"a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"
Jul 17 06:27:40.060533 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:40.060506    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-69mc9\" (UID: \"a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"
Jul 17 06:27:40.252929 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:40.252860    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"
Jul 17 06:27:40.386586 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:40.386562    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"]
Jul 17 06:27:40.388853 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:27:40.388825    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9c4e3ee_3eeb_4f8a_945e_6f417bf5fe85.slice/crio-34c631d1a315e9d023620b596e322f8641f7466ab11961453d5ae5c5f52acb89 WatchSource:0}: Error finding container 34c631d1a315e9d023620b596e322f8641f7466ab11961453d5ae5c5f52acb89: Status 404 returned error can't find the container with id 34c631d1a315e9d023620b596e322f8641f7466ab11961453d5ae5c5f52acb89
Jul 17 06:27:40.942575 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:40.942528    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9" event={"ID":"a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85","Type":"ContainerStarted","Data":"a4dcd9aa561732dfa839fc55394c54d31e8c81098abbaacb213bd58d5934c828"}
Jul 17 06:27:40.942967 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:40.942604    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9" event={"ID":"a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85","Type":"ContainerStarted","Data":"34c631d1a315e9d023620b596e322f8641f7466ab11961453d5ae5c5f52acb89"}
Jul 17 06:27:40.960869 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:40.960821    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9" podStartSLOduration=1.960801263 podStartE2EDuration="1.960801263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:27:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:27:40.960508345 +0000 UTC m=+2783.402446922" watchObservedRunningTime="2026-07-17 06:27:40.960801263 +0000 UTC m=+2783.402739838"
Jul 17 06:27:41.950326 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:41.950288    2558 generic.go:356] "Generic (PLEG): container finished" podID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerID="a4dcd9aa561732dfa839fc55394c54d31e8c81098abbaacb213bd58d5934c828" exitCode=6
Jul 17 06:27:41.950755 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:41.950431    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9" event={"ID":"a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85","Type":"ContainerDied","Data":"a4dcd9aa561732dfa839fc55394c54d31e8c81098abbaacb213bd58d5934c828"}
Jul 17 06:27:41.950755 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:41.950570    2558 scope.go:117] "RemoveContainer" containerID="a4dcd9aa561732dfa839fc55394c54d31e8c81098abbaacb213bd58d5934c828"
Jul 17 06:27:42.958717 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:42.958672    2558 generic.go:356] "Generic (PLEG): container finished" podID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerID="d9e1e9552098a30ae84752cd5985d3c8e154ce38b31c5d7857824231b0a7cb32" exitCode=6
Jul 17 06:27:42.959183 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:42.958763    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9" event={"ID":"a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85","Type":"ContainerDied","Data":"d9e1e9552098a30ae84752cd5985d3c8e154ce38b31c5d7857824231b0a7cb32"}
Jul 17 06:27:42.959183 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:42.958837    2558 scope.go:117] "RemoveContainer" containerID="a4dcd9aa561732dfa839fc55394c54d31e8c81098abbaacb213bd58d5934c828"
Jul 17 06:27:42.959183 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:42.959093    2558 scope.go:117] "RemoveContainer" containerID="d9e1e9552098a30ae84752cd5985d3c8e154ce38b31c5d7857824231b0a7cb32"
Jul 17 06:27:42.959360 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:42.959318    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-69mc9_odh-ai-gateway-infra(a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85"
Jul 17 06:27:43.965877 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:43.965845    2558 scope.go:117] "RemoveContainer" containerID="d9e1e9552098a30ae84752cd5985d3c8e154ce38b31c5d7857824231b0a7cb32"
Jul 17 06:27:43.966275 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:43.966078    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-69mc9_odh-ai-gateway-infra(a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85"
Jul 17 06:27:44.005971 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.005930    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"]
Jul 17 06:27:44.006436 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.006421    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerName="revoke-keys"
Jul 17 06:27:44.006508 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.006438    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e1a23b-646f-436c-bf46-c95b9fd913f9" containerName="revoke-keys"
Jul 17 06:27:44.006551 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.006527    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="274c2204-548b-4679-9dcc-d0842260bc46" containerName="revoke-keys"
Jul 17 06:27:44.010803 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.010785    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"
Jul 17 06:27:44.020228 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.020200    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"]
Jul 17 06:27:44.102236 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.102186    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr\" (UID: \"2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"
Jul 17 06:27:44.203297 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.203255    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr\" (UID: \"2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"
Jul 17 06:27:44.203964 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.203941    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr\" (UID: \"2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"
Jul 17 06:27:44.322880 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.322788    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"
Jul 17 06:27:44.488320 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.488298    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"]
Jul 17 06:27:44.490146 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:27:44.490119    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d57de21_94bd_4774_9ed4_c2a4cc3a0ad3.slice/crio-27cac02c2f9eb640ae5d112b7798586bae97459e5b1c12920d3b9950f66e649f WatchSource:0}: Error finding container 27cac02c2f9eb640ae5d112b7798586bae97459e5b1c12920d3b9950f66e649f: Status 404 returned error can't find the container with id 27cac02c2f9eb640ae5d112b7798586bae97459e5b1c12920d3b9950f66e649f
Jul 17 06:27:44.973600 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.973570    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr" event={"ID":"2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3","Type":"ContainerStarted","Data":"9fd6c8ec53906ec820400c6df299ea13e2c808a9e6120a292ef0433cc6ec2480"}
Jul 17 06:27:44.973948 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:44.973646    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr" event={"ID":"2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3","Type":"ContainerStarted","Data":"27cac02c2f9eb640ae5d112b7798586bae97459e5b1c12920d3b9950f66e649f"}
Jul 17 06:27:45.003091 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:45.003000    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr" podStartSLOduration=2.002984311 podStartE2EDuration="2.002984311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:27:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:27:45.000318847 +0000 UTC m=+2787.442257414" watchObservedRunningTime="2026-07-17 06:27:45.002984311 +0000 UTC m=+2787.444922877"
Jul 17 06:27:45.980550 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:45.980512    2558 generic.go:356] "Generic (PLEG): container finished" podID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerID="9fd6c8ec53906ec820400c6df299ea13e2c808a9e6120a292ef0433cc6ec2480" exitCode=6
Jul 17 06:27:45.980995 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:45.980602    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr" event={"ID":"2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3","Type":"ContainerDied","Data":"9fd6c8ec53906ec820400c6df299ea13e2c808a9e6120a292ef0433cc6ec2480"}
Jul 17 06:27:45.980995 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:45.980725    2558 scope.go:117] "RemoveContainer" containerID="9fd6c8ec53906ec820400c6df299ea13e2c808a9e6120a292ef0433cc6ec2480"
Jul 17 06:27:46.988390 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:46.988295    2558 generic.go:356] "Generic (PLEG): container finished" podID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerID="02a1ae429f68e21146c5ae746ac212e451f9649d41e757f410633b60d0d88cc1" exitCode=6
Jul 17 06:27:46.988887 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:46.988385    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr" event={"ID":"2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3","Type":"ContainerDied","Data":"02a1ae429f68e21146c5ae746ac212e451f9649d41e757f410633b60d0d88cc1"}
Jul 17 06:27:46.988887 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:46.988455    2558 scope.go:117] "RemoveContainer" containerID="9fd6c8ec53906ec820400c6df299ea13e2c808a9e6120a292ef0433cc6ec2480"
Jul 17 06:27:46.988887 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:46.988660    2558 scope.go:117] "RemoveContainer" containerID="02a1ae429f68e21146c5ae746ac212e451f9649d41e757f410633b60d0d88cc1"
Jul 17 06:27:46.989012 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:46.988921    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr_odh-ai-gateway-infra(2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3"
Jul 17 06:27:47.996137 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:47.996110    2558 scope.go:117] "RemoveContainer" containerID="02a1ae429f68e21146c5ae746ac212e451f9649d41e757f410633b60d0d88cc1"
Jul 17 06:27:47.996570 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:47.996355    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr_odh-ai-gateway-infra(2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3"
Jul 17 06:27:57.072149 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:57.072116    2558 scope.go:117] "RemoveContainer" containerID="d9e1e9552098a30ae84752cd5985d3c8e154ce38b31c5d7857824231b0a7cb32"
Jul 17 06:27:58.056430 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:58.056328    2558 generic.go:356] "Generic (PLEG): container finished" podID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerID="7fd9a50c75b82939fd53694e078ca8859acea4f3a3ecaf451a686e821eb90a89" exitCode=6
Jul 17 06:27:58.056430 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:58.056403    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9" event={"ID":"a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85","Type":"ContainerDied","Data":"7fd9a50c75b82939fd53694e078ca8859acea4f3a3ecaf451a686e821eb90a89"}
Jul 17 06:27:58.056679 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:58.056483    2558 scope.go:117] "RemoveContainer" containerID="d9e1e9552098a30ae84752cd5985d3c8e154ce38b31c5d7857824231b0a7cb32"
Jul 17 06:27:58.056857 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:58.056837    2558 scope.go:117] "RemoveContainer" containerID="7fd9a50c75b82939fd53694e078ca8859acea4f3a3ecaf451a686e821eb90a89"
Jul 17 06:27:58.057138 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:27:58.057108    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-69mc9_odh-ai-gateway-infra(a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85"
Jul 17 06:27:59.085699 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:59.085661    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"]
Jul 17 06:27:59.234559 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:59.234533    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"
Jul 17 06:27:59.351828 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:59.351730    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85-maas-api-service-ca\") pod \"a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85\" (UID: \"a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85\") "
Jul 17 06:27:59.352080 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:59.352055    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" (UID: "a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:27:59.453499 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:27:59.453440    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:28:00.069329 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:00.069295    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9" event={"ID":"a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85","Type":"ContainerDied","Data":"34c631d1a315e9d023620b596e322f8641f7466ab11961453d5ae5c5f52acb89"}
Jul 17 06:28:00.069329 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:00.069330    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"
Jul 17 06:28:00.069600 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:00.069361    2558 scope.go:117] "RemoveContainer" containerID="7fd9a50c75b82939fd53694e078ca8859acea4f3a3ecaf451a686e821eb90a89"
Jul 17 06:28:00.092156 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:00.092125    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"]
Jul 17 06:28:00.099931 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:00.099904    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-69mc9"]
Jul 17 06:28:01.073181 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:01.073153    2558 scope.go:117] "RemoveContainer" containerID="02a1ae429f68e21146c5ae746ac212e451f9649d41e757f410633b60d0d88cc1"
Jul 17 06:28:02.079053 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.078964    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" path="/var/lib/kubelet/pods/a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85/volumes"
Jul 17 06:28:02.082693 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.082668    2558 generic.go:356] "Generic (PLEG): container finished" podID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerID="0819feb085c151ed0ae1b04ca4449d4d19281632ad75a5f9aff542e6d8930a1a" exitCode=6
Jul 17 06:28:02.082829 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.082741    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr" event={"ID":"2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3","Type":"ContainerDied","Data":"0819feb085c151ed0ae1b04ca4449d4d19281632ad75a5f9aff542e6d8930a1a"}
Jul 17 06:28:02.082829 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.082795    2558 scope.go:117] "RemoveContainer" containerID="02a1ae429f68e21146c5ae746ac212e451f9649d41e757f410633b60d0d88cc1"
Jul 17 06:28:02.083084 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.083066    2558 scope.go:117] "RemoveContainer" containerID="0819feb085c151ed0ae1b04ca4449d4d19281632ad75a5f9aff542e6d8930a1a"
Jul 17 06:28:02.083320 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:28:02.083300    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr_odh-ai-gateway-infra(2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3"
Jul 17 06:28:02.961983 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.961952    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"]
Jul 17 06:28:02.962373 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.962361    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerName="revoke-keys"
Jul 17 06:28:02.962419 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.962375    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerName="revoke-keys"
Jul 17 06:28:02.962419 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.962385    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerName="revoke-keys"
Jul 17 06:28:02.962419 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.962390    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerName="revoke-keys"
Jul 17 06:28:02.962534 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.962461    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerName="revoke-keys"
Jul 17 06:28:02.962534 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.962487    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerName="revoke-keys"
Jul 17 06:28:02.966941 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.966922    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:02.975719 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.975697    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-bd7a398a-openshift-default-dockercfg-76jth\""
Jul 17 06:28:02.988345 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:02.988322    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"]
Jul 17 06:28:03.086204 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.086168    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-socket\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.086714 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.086222    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-podinfo\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.086714 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.086244    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dc3f20a-0089-4736-8828-14fa780c9c2d-istiod-ca-cert\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.086714 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.086321    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-credential-socket\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.086714 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.086423    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-token\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.086714 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.086454    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-certs\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.086714 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.086510    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-data\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.086714 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.086571    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzp7t\" (UniqueName: \"kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-kube-api-access-tzp7t\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.086714 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.086610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-envoy\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.108905 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.108865    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"]
Jul 17 06:28:03.188099 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.188064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-token\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.188288 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.188113    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-certs\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.188288 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.188137    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-data\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.188288 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.188190    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzp7t\" (UniqueName: \"kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-kube-api-access-tzp7t\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.188288 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.188228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-envoy\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.188565 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.188307    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-socket\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.188565 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.188363    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-podinfo\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.188565 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.188391    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dc3f20a-0089-4736-8828-14fa780c9c2d-istiod-ca-cert\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.188565 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.188417    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-credential-socket\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.188787 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.188651    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-certs\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.188787 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.188741    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-socket\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.189224 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.189179    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dc3f20a-0089-4736-8828-14fa780c9c2d-istiod-ca-cert\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.189224 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.189187    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-credential-socket\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.189413 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.189357    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-data\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.191600 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.191572    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-envoy\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.191727 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.191705    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-podinfo\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.197737 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.197706    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzp7t\" (UniqueName: \"kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-kube-api-access-tzp7t\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.198083 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.198064    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-token\") pod \"e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") " pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.276970 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.276943    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:03.282892 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.282872    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"
Jul 17 06:28:03.390599 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.390567    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3-maas-api-service-ca\") pod \"2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3\" (UID: \"2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3\") "
Jul 17 06:28:03.390986 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.390965    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" (UID: "2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:28:03.440384 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.440357    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"]
Jul 17 06:28:03.442202 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:28:03.442170    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dc3f20a_0089_4736_8828_14fa780c9c2d.slice/crio-6b1fef00504a9dc6fe04cd74ac8ec2034b71a559ff3ad6e3203d4ec11c835acb WatchSource:0}: Error finding container 6b1fef00504a9dc6fe04cd74ac8ec2034b71a559ff3ad6e3203d4ec11c835acb: Status 404 returned error can't find the container with id 6b1fef00504a9dc6fe04cd74ac8ec2034b71a559ff3ad6e3203d4ec11c835acb
Jul 17 06:28:03.444615 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.444581    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:28:03.444705 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.444652    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:28:03.444705 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.444681    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:28:03.491790 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:03.491759    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:28:04.098123 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:04.098082    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" event={"ID":"7dc3f20a-0089-4736-8828-14fa780c9c2d","Type":"ContainerStarted","Data":"b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500"}
Jul 17 06:28:04.098587 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:04.098152    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" event={"ID":"7dc3f20a-0089-4736-8828-14fa780c9c2d","Type":"ContainerStarted","Data":"6b1fef00504a9dc6fe04cd74ac8ec2034b71a559ff3ad6e3203d4ec11c835acb"}
Jul 17 06:28:04.099498 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:04.099466    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"
Jul 17 06:28:04.099558 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:04.099509    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr" event={"ID":"2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3","Type":"ContainerDied","Data":"27cac02c2f9eb640ae5d112b7798586bae97459e5b1c12920d3b9950f66e649f"}
Jul 17 06:28:04.099596 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:04.099566    2558 scope.go:117] "RemoveContainer" containerID="0819feb085c151ed0ae1b04ca4449d4d19281632ad75a5f9aff542e6d8930a1a"
Jul 17 06:28:04.154245 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:04.154189    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" podStartSLOduration=2.154168426 podStartE2EDuration="2.154168426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:28:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:28:04.145435522 +0000 UTC m=+2806.587374108" watchObservedRunningTime="2026-07-17 06:28:04.154168426 +0000 UTC m=+2806.596107000"
Jul 17 06:28:04.185601 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:04.185568    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"]
Jul 17 06:28:04.192884 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:04.192854    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hrvlr"]
Jul 17 06:28:04.277846 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:04.277813    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:04.283430 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:04.283402    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:05.108962 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:05.108931    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:05.110064 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:05.110042    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:06.079338 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:06.079305    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" path="/var/lib/kubelet/pods/2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3/volumes"
Jul 17 06:28:18.189224 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:18.189192    2558 generic.go:356] "Generic (PLEG): container finished" podID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerID="b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500" exitCode=0
Jul 17 06:28:18.189635 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:18.189260    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" event={"ID":"7dc3f20a-0089-4736-8828-14fa780c9c2d","Type":"ContainerDied","Data":"b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500"}
Jul 17 06:28:18.189695 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:18.189681    2558 scope.go:117] "RemoveContainer" containerID="b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500"
Jul 17 06:28:18.190708 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:18.190680    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:28:18.190768 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:18.190752    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:28:18.190804 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:18.190786    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:28:19.196879 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:19.196783    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" event={"ID":"7dc3f20a-0089-4736-8828-14fa780c9c2d","Type":"ContainerStarted","Data":"e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1"}
Jul 17 06:28:19.277955 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:19.277919    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:19.279048 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:19.279020    2558 patch_prober.go:28] interesting pod/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 17 06:28:19.279194 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:19.279076    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 17 06:28:20.109585 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:20.109558    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:20.277670 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:20.277638    2558 patch_prober.go:28] interesting pod/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 17 06:28:20.278048 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:20.277700    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 17 06:28:21.277688 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:21.277656    2558 patch_prober.go:28] interesting pod/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 17 06:28:21.278089 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:21.277709    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 17 06:28:22.278165 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:22.278125    2558 patch_prober.go:28] interesting pod/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 17 06:28:22.278588 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:22.278185    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 17 06:28:23.281953 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:23.281922    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:23.283174 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:23.283151    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:39.609594 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:39.609543    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"]
Jul 17 06:28:39.610170 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:39.610096    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerName="istio-proxy" containerID="cri-o://e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1" gracePeriod=30
Jul 17 06:28:44.875444 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.875418    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:44.902883 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.902825    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-certs\") pod \"7dc3f20a-0089-4736-8828-14fa780c9c2d\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") "
Jul 17 06:28:44.902883 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.902878    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-credential-socket\") pod \"7dc3f20a-0089-4736-8828-14fa780c9c2d\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") "
Jul 17 06:28:44.903069 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.902911    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dc3f20a-0089-4736-8828-14fa780c9c2d-istiod-ca-cert\") pod \"7dc3f20a-0089-4736-8828-14fa780c9c2d\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") "
Jul 17 06:28:44.903069 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.902969    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzp7t\" (UniqueName: \"kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-kube-api-access-tzp7t\") pod \"7dc3f20a-0089-4736-8828-14fa780c9c2d\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") "
Jul 17 06:28:44.903069 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.902993    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-data\") pod \"7dc3f20a-0089-4736-8828-14fa780c9c2d\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") "
Jul 17 06:28:44.903239 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.903100    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7dc3f20a-0089-4736-8828-14fa780c9c2d" (UID: "7dc3f20a-0089-4736-8828-14fa780c9c2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:28:44.903239 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.903151    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-socket\") pod \"7dc3f20a-0089-4736-8828-14fa780c9c2d\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") "
Jul 17 06:28:44.903344 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.903268    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7dc3f20a-0089-4736-8828-14fa780c9c2d" (UID: "7dc3f20a-0089-4736-8828-14fa780c9c2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:28:44.903344 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.903302    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dc3f20a-0089-4736-8828-14fa780c9c2d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7dc3f20a-0089-4736-8828-14fa780c9c2d" (UID: "7dc3f20a-0089-4736-8828-14fa780c9c2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:28:44.903512 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.903496    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:28:44.903575 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.903518    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:28:44.903575 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.903527    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dc3f20a-0089-4736-8828-14fa780c9c2d-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:28:44.903575 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.903493    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7dc3f20a-0089-4736-8828-14fa780c9c2d" (UID: "7dc3f20a-0089-4736-8828-14fa780c9c2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:28:44.903968 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.903941    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-data" (OuterVolumeSpecName: "istio-data") pod "7dc3f20a-0089-4736-8828-14fa780c9c2d" (UID: "7dc3f20a-0089-4736-8828-14fa780c9c2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:28:44.905229 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:44.905204    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-kube-api-access-tzp7t" (OuterVolumeSpecName: "kube-api-access-tzp7t") pod "7dc3f20a-0089-4736-8828-14fa780c9c2d" (UID: "7dc3f20a-0089-4736-8828-14fa780c9c2d"). InnerVolumeSpecName "kube-api-access-tzp7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:28:45.003881 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.003857    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-envoy\") pod \"7dc3f20a-0089-4736-8828-14fa780c9c2d\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") "
Jul 17 06:28:45.003989 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.003887    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-podinfo\") pod \"7dc3f20a-0089-4736-8828-14fa780c9c2d\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") "
Jul 17 06:28:45.003989 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.003904    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-token\") pod \"7dc3f20a-0089-4736-8828-14fa780c9c2d\" (UID: \"7dc3f20a-0089-4736-8828-14fa780c9c2d\") "
Jul 17 06:28:45.004121 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.004070    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzp7t\" (UniqueName: \"kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-kube-api-access-tzp7t\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:28:45.004121 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.004088    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:28:45.004121 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.004102    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:28:45.006018 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.005994    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7dc3f20a-0089-4736-8828-14fa780c9c2d" (UID: "7dc3f20a-0089-4736-8828-14fa780c9c2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:28:45.006018 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.006010    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7dc3f20a-0089-4736-8828-14fa780c9c2d" (UID: "7dc3f20a-0089-4736-8828-14fa780c9c2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:28:45.006167 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.006024    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-token" (OuterVolumeSpecName: "istio-token") pod "7dc3f20a-0089-4736-8828-14fa780c9c2d" (UID: "7dc3f20a-0089-4736-8828-14fa780c9c2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:28:45.104516 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.104464    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:28:45.104516 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.104513    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:28:45.104636 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.104522    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dc3f20a-0089-4736-8828-14fa780c9c2d-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:28:45.335057 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.335030    2558 generic.go:356] "Generic (PLEG): container finished" podID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerID="e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1" exitCode=0
Jul 17 06:28:45.335159 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.335080    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"
Jul 17 06:28:45.335159 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.335106    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" event={"ID":"7dc3f20a-0089-4736-8828-14fa780c9c2d","Type":"ContainerDied","Data":"e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1"}
Jul 17 06:28:45.335234 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.335162    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq" event={"ID":"7dc3f20a-0089-4736-8828-14fa780c9c2d","Type":"ContainerDied","Data":"6b1fef00504a9dc6fe04cd74ac8ec2034b71a559ff3ad6e3203d4ec11c835acb"}
Jul 17 06:28:45.335234 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.335186    2558 scope.go:117] "RemoveContainer" containerID="e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1"
Jul 17 06:28:45.344713 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.344690    2558 scope.go:117] "RemoveContainer" containerID="b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500"
Jul 17 06:28:45.352841 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.352826    2558 scope.go:117] "RemoveContainer" containerID="e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1"
Jul 17 06:28:45.353083 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:28:45.353067    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1\": container with ID starting with e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1 not found: ID does not exist" containerID="e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1"
Jul 17 06:28:45.353147 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.353088    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1"} err="failed to get container status \"e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1\": rpc error: code = NotFound desc = could not find container \"e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1\": container with ID starting with e1a5d7f63d0ad07e3b94402fd5c03f28bce2bb75d2048b16a61cea23c34f75b1 not found: ID does not exist"
Jul 17 06:28:45.353147 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.353103    2558 scope.go:117] "RemoveContainer" containerID="b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500"
Jul 17 06:28:45.353330 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:28:45.353312    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500\": container with ID starting with b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500 not found: ID does not exist" containerID="b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500"
Jul 17 06:28:45.353375 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.353338    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500"} err="failed to get container status \"b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500\": rpc error: code = NotFound desc = could not find container \"b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500\": container with ID starting with b3719a66b60f0bdca61f3b8b567b6870085319e141df2889f1d20df605619500 not found: ID does not exist"
Jul 17 06:28:45.356403 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.356384    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"]
Jul 17 06:28:45.363152 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:45.363125    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bd7a398a-openshift-default-5d59dd4dbc-fjcnq"]
Jul 17 06:28:46.079450 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:46.079408    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" path="/var/lib/kubelet/pods/7dc3f20a-0089-4736-8828-14fa780c9c2d/volumes"
Jul 17 06:28:58.872230 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872197    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"]
Jul 17 06:28:58.872684 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872668    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerName="revoke-keys"
Jul 17 06:28:58.872734 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872686    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerName="revoke-keys"
Jul 17 06:28:58.872734 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872697    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerName="revoke-keys"
Jul 17 06:28:58.872734 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872703    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerName="revoke-keys"
Jul 17 06:28:58.872734 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872722    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerName="istio-proxy"
Jul 17 06:28:58.872734 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872727    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerName="istio-proxy"
Jul 17 06:28:58.872886 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872738    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerName="revoke-keys"
Jul 17 06:28:58.872886 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872748    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerName="revoke-keys"
Jul 17 06:28:58.872886 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872755    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerName="istio-proxy"
Jul 17 06:28:58.872886 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872760    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerName="istio-proxy"
Jul 17 06:28:58.872886 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872829    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerName="revoke-keys"
Jul 17 06:28:58.872886 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872837    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerName="istio-proxy"
Jul 17 06:28:58.872886 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872842    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9c4e3ee-3eeb-4f8a-945e-6f417bf5fe85" containerName="revoke-keys"
Jul 17 06:28:58.872886 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872850    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerName="revoke-keys"
Jul 17 06:28:58.872886 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872857    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerName="revoke-keys"
Jul 17 06:28:58.872886 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.872864    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dc3f20a-0089-4736-8828-14fa780c9c2d" containerName="istio-proxy"
Jul 17 06:28:58.875013 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.874998    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:58.879790 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.879768    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3b164bbd-openshift-default-dockercfg-v56bc\""
Jul 17 06:28:58.900881 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.900856    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"]
Jul 17 06:28:58.908049 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.908025    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a936723f-8eca-467c-a3c4-5e868501e853-istiod-ca-cert\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:58.908192 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.908087    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-certs\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:58.908192 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.908113    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-envoy\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:58.908192 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.908146    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-data\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:58.908315 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.908228    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xwkn\" (UniqueName: \"kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-kube-api-access-7xwkn\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:58.908315 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.908267    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-credential-socket\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:58.908315 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.908292    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-socket\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:58.908426 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.908335    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-istio-token\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:58.908426 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:58.908358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a936723f-8eca-467c-a3c4-5e868501e853-istio-podinfo\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.009194 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.009161    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-istio-token\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.009363 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.009199    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a936723f-8eca-467c-a3c4-5e868501e853-istio-podinfo\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.009363 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.009331    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a936723f-8eca-467c-a3c4-5e868501e853-istiod-ca-cert\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.009453 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.009389    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-certs\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.009453 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.009408    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-envoy\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.009453 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.009435    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-data\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.009655 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.009462    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xwkn\" (UniqueName: \"kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-kube-api-access-7xwkn\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.009655 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.009520    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-credential-socket\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.009655 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.009560    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-socket\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.009838 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.009810    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-certs\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.010118 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.010093    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a936723f-8eca-467c-a3c4-5e868501e853-istiod-ca-cert\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.010242 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.010102    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-socket\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.010242 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.010108    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-data\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.010242 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.010152    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-credential-socket\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.011815 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.011795    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a936723f-8eca-467c-a3c4-5e868501e853-istio-podinfo\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.011907 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.011838    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-envoy\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.026713 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.026682    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-istio-token\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.030307 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.030279    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xwkn\" (UniqueName: \"kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-kube-api-access-7xwkn\") pod \"e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") " pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.186185 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.186113    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:28:59.332024 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.331996    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"]
Jul 17 06:28:59.333727 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:28:59.333698    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda936723f_8eca_467c_a3c4_5e868501e853.slice/crio-da52eff231cffef04a2a4248f75716d052aa014f635b7a8c6c6d0dd518024854 WatchSource:0}: Error finding container da52eff231cffef04a2a4248f75716d052aa014f635b7a8c6c6d0dd518024854: Status 404 returned error can't find the container with id da52eff231cffef04a2a4248f75716d052aa014f635b7a8c6c6d0dd518024854
Jul 17 06:28:59.336090 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.336060    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:28:59.336179 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.336121    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:28:59.336179 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.336153    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:28:59.411214 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.411184    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" event={"ID":"a936723f-8eca-467c-a3c4-5e868501e853","Type":"ContainerStarted","Data":"575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259"}
Jul 17 06:28:59.411346 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.411238    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" event={"ID":"a936723f-8eca-467c-a3c4-5e868501e853","Type":"ContainerStarted","Data":"da52eff231cffef04a2a4248f75716d052aa014f635b7a8c6c6d0dd518024854"}
Jul 17 06:28:59.433739 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:28:59.433688    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" podStartSLOduration=1.433670841 podStartE2EDuration="1.433670841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:28:59.43123893 +0000 UTC m=+2861.873177506" watchObservedRunningTime="2026-07-17 06:28:59.433670841 +0000 UTC m=+2861.875609416"
Jul 17 06:29:00.187025 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:00.186996    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:29:00.192036 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:00.192009    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:29:00.416751 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:00.416720    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:29:00.417545 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:00.417527    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:29:12.441063 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:12.441027    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"]
Jul 17 06:29:12.441507 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:12.441466    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerName="revoke-keys"
Jul 17 06:29:12.441507 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:12.441494    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d57de21-94bd-4774-9ed4-c2a4cc3a0ad3" containerName="revoke-keys"
Jul 17 06:29:12.445151 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:12.445132    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"
Jul 17 06:29:12.448043 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:12.448013    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:29:12.455334 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:12.455309    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"]
Jul 17 06:29:12.528620 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:12.528569    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/795eddde-6d2f-48fe-b729-aaa78ec2b234-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-78t2l\" (UID: \"795eddde-6d2f-48fe-b729-aaa78ec2b234\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"
Jul 17 06:29:12.629764 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:12.629721    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/795eddde-6d2f-48fe-b729-aaa78ec2b234-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-78t2l\" (UID: \"795eddde-6d2f-48fe-b729-aaa78ec2b234\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"
Jul 17 06:29:12.630548 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:12.630522    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/795eddde-6d2f-48fe-b729-aaa78ec2b234-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-78t2l\" (UID: \"795eddde-6d2f-48fe-b729-aaa78ec2b234\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"
Jul 17 06:29:12.759703 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:12.759607    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"
Jul 17 06:29:12.911604 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:12.911579    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"]
Jul 17 06:29:12.914168 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:29:12.914140    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795eddde_6d2f_48fe_b729_aaa78ec2b234.slice/crio-a7847d3a6ceb5b7e589ab78b9e4be93808207fe9a0fbe9cb656528419db7e1d8 WatchSource:0}: Error finding container a7847d3a6ceb5b7e589ab78b9e4be93808207fe9a0fbe9cb656528419db7e1d8: Status 404 returned error can't find the container with id a7847d3a6ceb5b7e589ab78b9e4be93808207fe9a0fbe9cb656528419db7e1d8
Jul 17 06:29:13.490073 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:13.489992    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l" event={"ID":"795eddde-6d2f-48fe-b729-aaa78ec2b234","Type":"ContainerStarted","Data":"23764d4024b11b0aaa7d2726d7943cb068e4c07552e47d0f2bba48c508dbd608"}
Jul 17 06:29:13.490073 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:13.490046    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l" event={"ID":"795eddde-6d2f-48fe-b729-aaa78ec2b234","Type":"ContainerStarted","Data":"a7847d3a6ceb5b7e589ab78b9e4be93808207fe9a0fbe9cb656528419db7e1d8"}
Jul 17 06:29:13.507394 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:13.507343    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l" podStartSLOduration=1.507324765 podStartE2EDuration="1.507324765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:29:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:29:13.506056942 +0000 UTC m=+2875.947995517" watchObservedRunningTime="2026-07-17 06:29:13.507324765 +0000 UTC m=+2875.949263340"
Jul 17 06:29:14.496384 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:14.496354    2558 generic.go:356] "Generic (PLEG): container finished" podID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerID="23764d4024b11b0aaa7d2726d7943cb068e4c07552e47d0f2bba48c508dbd608" exitCode=6
Jul 17 06:29:14.496780 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:14.496450    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l" event={"ID":"795eddde-6d2f-48fe-b729-aaa78ec2b234","Type":"ContainerDied","Data":"23764d4024b11b0aaa7d2726d7943cb068e4c07552e47d0f2bba48c508dbd608"}
Jul 17 06:29:14.496780 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:14.496606    2558 scope.go:117] "RemoveContainer" containerID="23764d4024b11b0aaa7d2726d7943cb068e4c07552e47d0f2bba48c508dbd608"
Jul 17 06:29:15.417725 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.417699    2558 patch_prober.go:28] interesting pod/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Jul 17 06:29:15.417854 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.417745    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Jul 17 06:29:15.503330 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.503290    2558 generic.go:356] "Generic (PLEG): container finished" podID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerID="90678f4f99808376a4d77ea31c52632ff778e7f68278098e34f75ed595b54d63" exitCode=6
Jul 17 06:29:15.503694 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.503353    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l" event={"ID":"795eddde-6d2f-48fe-b729-aaa78ec2b234","Type":"ContainerDied","Data":"90678f4f99808376a4d77ea31c52632ff778e7f68278098e34f75ed595b54d63"}
Jul 17 06:29:15.503694 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.503414    2558 scope.go:117] "RemoveContainer" containerID="23764d4024b11b0aaa7d2726d7943cb068e4c07552e47d0f2bba48c508dbd608"
Jul 17 06:29:15.503694 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.503601    2558 scope.go:117] "RemoveContainer" containerID="90678f4f99808376a4d77ea31c52632ff778e7f68278098e34f75ed595b54d63"
Jul 17 06:29:15.503839 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:29:15.503819    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-78t2l_odh-ai-gateway-infra(795eddde-6d2f-48fe-b729-aaa78ec2b234)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234"
Jul 17 06:29:15.505157 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.505131    2558 generic.go:356] "Generic (PLEG): container finished" podID="a936723f-8eca-467c-a3c4-5e868501e853" containerID="575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259" exitCode=0
Jul 17 06:29:15.505253 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.505176    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" event={"ID":"a936723f-8eca-467c-a3c4-5e868501e853","Type":"ContainerDied","Data":"575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259"}
Jul 17 06:29:15.505546 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.505532    2558 scope.go:117] "RemoveContainer" containerID="575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259"
Jul 17 06:29:15.506570 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.506539    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:29:15.506667 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.506612    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:29:15.506722 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:15.506669    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:29:16.512373 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:16.512336    2558 scope.go:117] "RemoveContainer" containerID="90678f4f99808376a4d77ea31c52632ff778e7f68278098e34f75ed595b54d63"
Jul 17 06:29:16.512821 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:29:16.512628    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-78t2l_odh-ai-gateway-infra(795eddde-6d2f-48fe-b729-aaa78ec2b234)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234"
Jul 17 06:29:16.514053 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:16.514028    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" event={"ID":"a936723f-8eca-467c-a3c4-5e868501e853","Type":"ContainerStarted","Data":"112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f"}
Jul 17 06:29:17.186910 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:17.186860    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:29:17.187184 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:17.187158    2558 patch_prober.go:28] interesting pod/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Jul 17 06:29:17.187252 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:17.187213    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Jul 17 06:29:18.187610 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:18.187565    2558 patch_prober.go:28] interesting pod/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Jul 17 06:29:18.188126 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:18.187631    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Jul 17 06:29:19.187201 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:19.187112    2558 patch_prober.go:28] interesting pod/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Jul 17 06:29:19.187201 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:19.187175    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Jul 17 06:29:20.187484 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:20.187448    2558 patch_prober.go:28] interesting pod/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Jul 17 06:29:20.187850 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:20.187538    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Jul 17 06:29:21.191339 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:21.191303    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:29:21.191825 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:21.191591    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:29:21.192207 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:21.192189    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:29:31.072528 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:31.072464    2558 scope.go:117] "RemoveContainer" containerID="90678f4f99808376a4d77ea31c52632ff778e7f68278098e34f75ed595b54d63"
Jul 17 06:29:31.601597 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:31.601512    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l" event={"ID":"795eddde-6d2f-48fe-b729-aaa78ec2b234","Type":"ContainerStarted","Data":"8cdf4ed03cf14871da4ca795d0a3844803bcf07cc85836897e1c7019c7979a8d"}
Jul 17 06:29:32.608813 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:32.608774    2558 generic.go:356] "Generic (PLEG): container finished" podID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerID="8cdf4ed03cf14871da4ca795d0a3844803bcf07cc85836897e1c7019c7979a8d" exitCode=6
Jul 17 06:29:32.609222 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:32.608845    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l" event={"ID":"795eddde-6d2f-48fe-b729-aaa78ec2b234","Type":"ContainerDied","Data":"8cdf4ed03cf14871da4ca795d0a3844803bcf07cc85836897e1c7019c7979a8d"}
Jul 17 06:29:32.609222 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:32.608914    2558 scope.go:117] "RemoveContainer" containerID="90678f4f99808376a4d77ea31c52632ff778e7f68278098e34f75ed595b54d63"
Jul 17 06:29:32.609332 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:32.609255    2558 scope.go:117] "RemoveContainer" containerID="8cdf4ed03cf14871da4ca795d0a3844803bcf07cc85836897e1c7019c7979a8d"
Jul 17 06:29:32.609530 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:29:32.609501    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-78t2l_odh-ai-gateway-infra(795eddde-6d2f-48fe-b729-aaa78ec2b234)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234"
Jul 17 06:29:32.643073 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:32.642979    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"]
Jul 17 06:29:33.762855 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:33.762832    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"
Jul 17 06:29:33.842908 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:33.842880    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/795eddde-6d2f-48fe-b729-aaa78ec2b234-maas-api-service-ca\") pod \"795eddde-6d2f-48fe-b729-aaa78ec2b234\" (UID: \"795eddde-6d2f-48fe-b729-aaa78ec2b234\") "
Jul 17 06:29:33.843228 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:33.843206    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/795eddde-6d2f-48fe-b729-aaa78ec2b234-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "795eddde-6d2f-48fe-b729-aaa78ec2b234" (UID: "795eddde-6d2f-48fe-b729-aaa78ec2b234"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:29:33.943898 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:33.943799    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/795eddde-6d2f-48fe-b729-aaa78ec2b234-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:29:34.623721 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:34.623685    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"
Jul 17 06:29:34.623936 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:34.623720    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l" event={"ID":"795eddde-6d2f-48fe-b729-aaa78ec2b234","Type":"ContainerDied","Data":"a7847d3a6ceb5b7e589ab78b9e4be93808207fe9a0fbe9cb656528419db7e1d8"}
Jul 17 06:29:34.623936 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:34.623788    2558 scope.go:117] "RemoveContainer" containerID="8cdf4ed03cf14871da4ca795d0a3844803bcf07cc85836897e1c7019c7979a8d"
Jul 17 06:29:34.640208 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:34.640175    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"]
Jul 17 06:29:34.645977 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:34.645944    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-78t2l"]
Jul 17 06:29:36.078953 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:36.078915    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234" path="/var/lib/kubelet/pods/795eddde-6d2f-48fe-b729-aaa78ec2b234/volumes"
Jul 17 06:29:40.051878 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.051828    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"]
Jul 17 06:29:40.052512 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.052467    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerName="revoke-keys"
Jul 17 06:29:40.052620 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.052514    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerName="revoke-keys"
Jul 17 06:29:40.052620 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.052540    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerName="revoke-keys"
Jul 17 06:29:40.052620 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.052549    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerName="revoke-keys"
Jul 17 06:29:40.052779 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.052678    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerName="revoke-keys"
Jul 17 06:29:40.052779 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.052693    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerName="revoke-keys"
Jul 17 06:29:40.056206 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.056183    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"
Jul 17 06:29:40.058781 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.058756    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:29:40.063636 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.063381    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"]
Jul 17 06:29:40.099951 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.099916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54d83c85-0c9c-415c-9a2b-7ee7c436cada-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd\" (UID: \"54d83c85-0c9c-415c-9a2b-7ee7c436cada\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"
Jul 17 06:29:40.200920 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.200886    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54d83c85-0c9c-415c-9a2b-7ee7c436cada-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd\" (UID: \"54d83c85-0c9c-415c-9a2b-7ee7c436cada\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"
Jul 17 06:29:40.201512 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.201493    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54d83c85-0c9c-415c-9a2b-7ee7c436cada-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd\" (UID: \"54d83c85-0c9c-415c-9a2b-7ee7c436cada\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"
Jul 17 06:29:40.370403 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.370374    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"
Jul 17 06:29:40.504808 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.504776    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"]
Jul 17 06:29:40.506459 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:29:40.506430    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d83c85_0c9c_415c_9a2b_7ee7c436cada.slice/crio-d5939eb276ea237c2847dc45c190563fd7ba4949c62bf017acf2b058b3bd32dc WatchSource:0}: Error finding container d5939eb276ea237c2847dc45c190563fd7ba4949c62bf017acf2b058b3bd32dc: Status 404 returned error can't find the container with id d5939eb276ea237c2847dc45c190563fd7ba4949c62bf017acf2b058b3bd32dc
Jul 17 06:29:40.657767 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:40.657730    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd" event={"ID":"54d83c85-0c9c-415c-9a2b-7ee7c436cada","Type":"ContainerStarted","Data":"d5939eb276ea237c2847dc45c190563fd7ba4949c62bf017acf2b058b3bd32dc"}
Jul 17 06:29:41.664447 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:41.664413    2558 generic.go:356] "Generic (PLEG): container finished" podID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerID="48f54ed7e949e05a8318ae59c7f94ded078301fa4958be46e126e6ced56727e2" exitCode=6
Jul 17 06:29:41.664883 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:41.664500    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd" event={"ID":"54d83c85-0c9c-415c-9a2b-7ee7c436cada","Type":"ContainerDied","Data":"48f54ed7e949e05a8318ae59c7f94ded078301fa4958be46e126e6ced56727e2"}
Jul 17 06:29:41.664883 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:41.664744    2558 scope.go:117] "RemoveContainer" containerID="48f54ed7e949e05a8318ae59c7f94ded078301fa4958be46e126e6ced56727e2"
Jul 17 06:29:42.672429 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:42.672390    2558 generic.go:356] "Generic (PLEG): container finished" podID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerID="8c25bf834ea3075d632bc2b200e91a4de1dc1d97cd1e94816574325f52790b43" exitCode=6
Jul 17 06:29:42.673115 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:42.672441    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd" event={"ID":"54d83c85-0c9c-415c-9a2b-7ee7c436cada","Type":"ContainerDied","Data":"8c25bf834ea3075d632bc2b200e91a4de1dc1d97cd1e94816574325f52790b43"}
Jul 17 06:29:42.673115 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:42.672522    2558 scope.go:117] "RemoveContainer" containerID="48f54ed7e949e05a8318ae59c7f94ded078301fa4958be46e126e6ced56727e2"
Jul 17 06:29:42.673115 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:42.672675    2558 scope.go:117] "RemoveContainer" containerID="8c25bf834ea3075d632bc2b200e91a4de1dc1d97cd1e94816574325f52790b43"
Jul 17 06:29:42.673115 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:29:42.672935    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd_odh-ai-gateway-infra(54d83c85-0c9c-415c-9a2b-7ee7c436cada)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada"
Jul 17 06:29:43.679876 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:43.679847    2558 scope.go:117] "RemoveContainer" containerID="8c25bf834ea3075d632bc2b200e91a4de1dc1d97cd1e94816574325f52790b43"
Jul 17 06:29:43.680350 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:29:43.680155    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd_odh-ai-gateway-infra(54d83c85-0c9c-415c-9a2b-7ee7c436cada)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada"
Jul 17 06:29:44.080593 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.080506    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"]
Jul 17 06:29:44.081006 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.080986    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerName="revoke-keys"
Jul 17 06:29:44.081102 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.081008    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerName="revoke-keys"
Jul 17 06:29:44.081159 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.081132    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="795eddde-6d2f-48fe-b729-aaa78ec2b234" containerName="revoke-keys"
Jul 17 06:29:44.083921 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.083897    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"
Jul 17 06:29:44.086023 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.085990    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"]
Jul 17 06:29:44.141159 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.141106    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq\" (UID: \"af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"
Jul 17 06:29:44.242062 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.242023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq\" (UID: \"af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"
Jul 17 06:29:44.242664 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.242645    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq\" (UID: \"af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"
Jul 17 06:29:44.396527 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.396487    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"
Jul 17 06:29:44.537958 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.537934    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"]
Jul 17 06:29:44.540309 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:29:44.540277    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf3ac34a_b8f4_44ff_86d8_8d6ec5cc8668.slice/crio-2e93cce84309600f0e620e6fa366501a9ab7ae9e84773b198141bedf70cee7b3 WatchSource:0}: Error finding container 2e93cce84309600f0e620e6fa366501a9ab7ae9e84773b198141bedf70cee7b3: Status 404 returned error can't find the container with id 2e93cce84309600f0e620e6fa366501a9ab7ae9e84773b198141bedf70cee7b3
Jul 17 06:29:44.686927 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:44.686833    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq" event={"ID":"af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668","Type":"ContainerStarted","Data":"2e93cce84309600f0e620e6fa366501a9ab7ae9e84773b198141bedf70cee7b3"}
Jul 17 06:29:45.693520 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:45.693484    2558 generic.go:356] "Generic (PLEG): container finished" podID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerID="ce07fe285f633d2008f3ddcd1a99e786efa89a57b8680a5b112b662b897cd4cf" exitCode=6
Jul 17 06:29:45.693925 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:45.693548    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq" event={"ID":"af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668","Type":"ContainerDied","Data":"ce07fe285f633d2008f3ddcd1a99e786efa89a57b8680a5b112b662b897cd4cf"}
Jul 17 06:29:45.693925 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:45.693817    2558 scope.go:117] "RemoveContainer" containerID="ce07fe285f633d2008f3ddcd1a99e786efa89a57b8680a5b112b662b897cd4cf"
Jul 17 06:29:46.700828 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:46.700746    2558 generic.go:356] "Generic (PLEG): container finished" podID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerID="d55dda8661c87f762398edb840a8d8211a69c72488464fba00e32cc2ed03af2d" exitCode=6
Jul 17 06:29:46.700828 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:46.700811    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq" event={"ID":"af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668","Type":"ContainerDied","Data":"d55dda8661c87f762398edb840a8d8211a69c72488464fba00e32cc2ed03af2d"}
Jul 17 06:29:46.701337 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:46.700859    2558 scope.go:117] "RemoveContainer" containerID="ce07fe285f633d2008f3ddcd1a99e786efa89a57b8680a5b112b662b897cd4cf"
Jul 17 06:29:46.701337 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:46.701097    2558 scope.go:117] "RemoveContainer" containerID="d55dda8661c87f762398edb840a8d8211a69c72488464fba00e32cc2ed03af2d"
Jul 17 06:29:46.701444 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:29:46.701339    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq_odh-ai-gateway-infra(af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668"
Jul 17 06:29:47.707872 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:47.707843    2558 scope.go:117] "RemoveContainer" containerID="d55dda8661c87f762398edb840a8d8211a69c72488464fba00e32cc2ed03af2d"
Jul 17 06:29:47.708361 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:29:47.708124    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq_odh-ai-gateway-infra(af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668"
Jul 17 06:29:59.072403 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:59.072369    2558 scope.go:117] "RemoveContainer" containerID="8c25bf834ea3075d632bc2b200e91a4de1dc1d97cd1e94816574325f52790b43"
Jul 17 06:29:59.782729 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:59.782690    2558 generic.go:356] "Generic (PLEG): container finished" podID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerID="a14e794fd7127dce541a61e2279c0c32635e8715b3f0ace273bae9501e8db5de" exitCode=6
Jul 17 06:29:59.782945 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:59.782772    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd" event={"ID":"54d83c85-0c9c-415c-9a2b-7ee7c436cada","Type":"ContainerDied","Data":"a14e794fd7127dce541a61e2279c0c32635e8715b3f0ace273bae9501e8db5de"}
Jul 17 06:29:59.782945 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:59.782864    2558 scope.go:117] "RemoveContainer" containerID="8c25bf834ea3075d632bc2b200e91a4de1dc1d97cd1e94816574325f52790b43"
Jul 17 06:29:59.783240 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:29:59.783224    2558 scope.go:117] "RemoveContainer" containerID="a14e794fd7127dce541a61e2279c0c32635e8715b3f0ace273bae9501e8db5de"
Jul 17 06:29:59.783525 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:29:59.783498    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd_odh-ai-gateway-infra(54d83c85-0c9c-415c-9a2b-7ee7c436cada)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada"
Jul 17 06:30:00.813459 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:00.813424    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"]
Jul 17 06:30:00.962832 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:00.962806    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"
Jul 17 06:30:01.103466 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:01.103417    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54d83c85-0c9c-415c-9a2b-7ee7c436cada-maas-api-service-ca\") pod \"54d83c85-0c9c-415c-9a2b-7ee7c436cada\" (UID: \"54d83c85-0c9c-415c-9a2b-7ee7c436cada\") "
Jul 17 06:30:01.103827 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:01.103803    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54d83c85-0c9c-415c-9a2b-7ee7c436cada-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54d83c85-0c9c-415c-9a2b-7ee7c436cada" (UID: "54d83c85-0c9c-415c-9a2b-7ee7c436cada"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:30:01.205235 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:01.205187    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54d83c85-0c9c-415c-9a2b-7ee7c436cada-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:30:01.801287 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:01.801255    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"
Jul 17 06:30:01.801526 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:01.801295    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd" event={"ID":"54d83c85-0c9c-415c-9a2b-7ee7c436cada","Type":"ContainerDied","Data":"d5939eb276ea237c2847dc45c190563fd7ba4949c62bf017acf2b058b3bd32dc"}
Jul 17 06:30:01.801526 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:01.801360    2558 scope.go:117] "RemoveContainer" containerID="a14e794fd7127dce541a61e2279c0c32635e8715b3f0ace273bae9501e8db5de"
Jul 17 06:30:01.826279 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:01.826249    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"]
Jul 17 06:30:01.831937 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:01.831909    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-zhzdd"]
Jul 17 06:30:02.079556 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:02.079452    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" path="/var/lib/kubelet/pods/54d83c85-0c9c-415c-9a2b-7ee7c436cada/volumes"
Jul 17 06:30:03.073068 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:03.073038    2558 scope.go:117] "RemoveContainer" containerID="d55dda8661c87f762398edb840a8d8211a69c72488464fba00e32cc2ed03af2d"
Jul 17 06:30:03.814373 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:03.814338    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq" event={"ID":"af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668","Type":"ContainerStarted","Data":"c4216d69584056ed37d858be5050496cc5a757388bcd0db2e61dcf89d316425d"}
Jul 17 06:30:03.829663 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:03.829597    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq" podStartSLOduration=19.829577785 podStartE2EDuration="19.829577785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:29:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:30:03.82915881 +0000 UTC m=+2926.271097376" watchObservedRunningTime="2026-07-17 06:30:03.829577785 +0000 UTC m=+2926.271516361"
Jul 17 06:30:04.821784 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:04.821741    2558 generic.go:356] "Generic (PLEG): container finished" podID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerID="c4216d69584056ed37d858be5050496cc5a757388bcd0db2e61dcf89d316425d" exitCode=6
Jul 17 06:30:04.822226 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:04.821806    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq" event={"ID":"af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668","Type":"ContainerDied","Data":"c4216d69584056ed37d858be5050496cc5a757388bcd0db2e61dcf89d316425d"}
Jul 17 06:30:04.822226 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:04.821876    2558 scope.go:117] "RemoveContainer" containerID="d55dda8661c87f762398edb840a8d8211a69c72488464fba00e32cc2ed03af2d"
Jul 17 06:30:04.822312 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:04.822256    2558 scope.go:117] "RemoveContainer" containerID="c4216d69584056ed37d858be5050496cc5a757388bcd0db2e61dcf89d316425d"
Jul 17 06:30:04.822552 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:30:04.822519    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq_odh-ai-gateway-infra(af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668"
Jul 17 06:30:04.839193 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:04.839164    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"]
Jul 17 06:30:05.974935 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:05.974909    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"
Jul 17 06:30:06.052939 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:06.052898    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668-maas-api-service-ca\") pod \"af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668\" (UID: \"af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668\") "
Jul 17 06:30:06.053321 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:06.053292    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" (UID: "af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:30:06.154627 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:06.154588    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:30:06.835562 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:06.835522    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"
Jul 17 06:30:06.835562 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:06.835543    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq" event={"ID":"af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668","Type":"ContainerDied","Data":"2e93cce84309600f0e620e6fa366501a9ab7ae9e84773b198141bedf70cee7b3"}
Jul 17 06:30:06.835804 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:06.835608    2558 scope.go:117] "RemoveContainer" containerID="c4216d69584056ed37d858be5050496cc5a757388bcd0db2e61dcf89d316425d"
Jul 17 06:30:06.852709 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:06.852684    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"]
Jul 17 06:30:06.858552 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:06.858531    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-hr2gq"]
Jul 17 06:30:08.079699 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:30:08.079660    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" path="/var/lib/kubelet/pods/af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668/volumes"
Jul 17 06:31:13.692701 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:13.692671    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"]
Jul 17 06:31:13.693259 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:13.692869    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy" containerID="cri-o://112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f" gracePeriod=30
Jul 17 06:31:18.233407 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.233367    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:31:18.238888 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.238853    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:31:18.239838 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.239811    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:31:18.245632 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.245613    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:31:18.962957 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.962890    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:31:18.996927 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.996897    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-socket\") pod \"a936723f-8eca-467c-a3c4-5e868501e853\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") "
Jul 17 06:31:18.997084 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.996964    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-envoy\") pod \"a936723f-8eca-467c-a3c4-5e868501e853\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") "
Jul 17 06:31:18.997084 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.996984    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-certs\") pod \"a936723f-8eca-467c-a3c4-5e868501e853\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") "
Jul 17 06:31:18.997084 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997000    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-istio-token\") pod \"a936723f-8eca-467c-a3c4-5e868501e853\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") "
Jul 17 06:31:18.997084 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997015    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xwkn\" (UniqueName: \"kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-kube-api-access-7xwkn\") pod \"a936723f-8eca-467c-a3c4-5e868501e853\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") "
Jul 17 06:31:18.997084 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997036    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a936723f-8eca-467c-a3c4-5e868501e853-istiod-ca-cert\") pod \"a936723f-8eca-467c-a3c4-5e868501e853\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") "
Jul 17 06:31:18.997084 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997054    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-credential-socket\") pod \"a936723f-8eca-467c-a3c4-5e868501e853\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") "
Jul 17 06:31:18.997405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997136    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a936723f-8eca-467c-a3c4-5e868501e853-istio-podinfo\") pod \"a936723f-8eca-467c-a3c4-5e868501e853\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") "
Jul 17 06:31:18.997405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997172    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-data\") pod \"a936723f-8eca-467c-a3c4-5e868501e853\" (UID: \"a936723f-8eca-467c-a3c4-5e868501e853\") "
Jul 17 06:31:18.997405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997187    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a936723f-8eca-467c-a3c4-5e868501e853" (UID: "a936723f-8eca-467c-a3c4-5e868501e853"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:31:18.997405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997301    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a936723f-8eca-467c-a3c4-5e868501e853" (UID: "a936723f-8eca-467c-a3c4-5e868501e853"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:31:18.997654 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997445    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a936723f-8eca-467c-a3c4-5e868501e853-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a936723f-8eca-467c-a3c4-5e868501e853" (UID: "a936723f-8eca-467c-a3c4-5e868501e853"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:31:18.997654 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997537    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:31:18.997654 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997602    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:31:18.997810 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.997788    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a936723f-8eca-467c-a3c4-5e868501e853" (UID: "a936723f-8eca-467c-a3c4-5e868501e853"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:31:18.998309 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.998282    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-data" (OuterVolumeSpecName: "istio-data") pod "a936723f-8eca-467c-a3c4-5e868501e853" (UID: "a936723f-8eca-467c-a3c4-5e868501e853"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:31:18.999539 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.999502    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a936723f-8eca-467c-a3c4-5e868501e853" (UID: "a936723f-8eca-467c-a3c4-5e868501e853"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:31:18.999666 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.999643    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a936723f-8eca-467c-a3c4-5e868501e853-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a936723f-8eca-467c-a3c4-5e868501e853" (UID: "a936723f-8eca-467c-a3c4-5e868501e853"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:31:18.999824 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.999798    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-istio-token" (OuterVolumeSpecName: "istio-token") pod "a936723f-8eca-467c-a3c4-5e868501e853" (UID: "a936723f-8eca-467c-a3c4-5e868501e853"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:31:18.999920 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:18.999870    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-kube-api-access-7xwkn" (OuterVolumeSpecName: "kube-api-access-7xwkn") pod "a936723f-8eca-467c-a3c4-5e868501e853" (UID: "a936723f-8eca-467c-a3c4-5e868501e853"). InnerVolumeSpecName "kube-api-access-7xwkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:31:19.101503 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.098628    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a936723f-8eca-467c-a3c4-5e868501e853-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:31:19.101503 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.098661    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:31:19.101503 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.098685    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:31:19.101503 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.098701    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:31:19.101503 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.098717    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xwkn\" (UniqueName: \"kubernetes.io/projected/a936723f-8eca-467c-a3c4-5e868501e853-kube-api-access-7xwkn\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:31:19.101503 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.098732    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a936723f-8eca-467c-a3c4-5e868501e853-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:31:19.101503 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.098752    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a936723f-8eca-467c-a3c4-5e868501e853-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:31:19.219912 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.219808    2558 generic.go:356] "Generic (PLEG): container finished" podID="a936723f-8eca-467c-a3c4-5e868501e853" containerID="112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f" exitCode=0
Jul 17 06:31:19.219912 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.219858    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" event={"ID":"a936723f-8eca-467c-a3c4-5e868501e853","Type":"ContainerDied","Data":"112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f"}
Jul 17 06:31:19.219912 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.219888    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"
Jul 17 06:31:19.220260 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.219930    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8" event={"ID":"a936723f-8eca-467c-a3c4-5e868501e853","Type":"ContainerDied","Data":"da52eff231cffef04a2a4248f75716d052aa014f635b7a8c6c6d0dd518024854"}
Jul 17 06:31:19.220260 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.219951    2558 scope.go:117] "RemoveContainer" containerID="112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f"
Jul 17 06:31:19.230447 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.230424    2558 scope.go:117] "RemoveContainer" containerID="575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259"
Jul 17 06:31:19.242969 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.242943    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"]
Jul 17 06:31:19.243281 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.243265    2558 scope.go:117] "RemoveContainer" containerID="112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f"
Jul 17 06:31:19.243618 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:31:19.243599    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f\": container with ID starting with 112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f not found: ID does not exist" containerID="112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f"
Jul 17 06:31:19.243675 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.243624    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f"} err="failed to get container status \"112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f\": rpc error: code = NotFound desc = could not find container \"112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f\": container with ID starting with 112bd1e18eb585d851a15016766d17ebee5ab772e576085c78e3a8d2142a455f not found: ID does not exist"
Jul 17 06:31:19.243675 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.243642    2558 scope.go:117] "RemoveContainer" containerID="575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259"
Jul 17 06:31:19.243881 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:31:19.243863    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259\": container with ID starting with 575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259 not found: ID does not exist" containerID="575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259"
Jul 17 06:31:19.243936 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.243888    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259"} err="failed to get container status \"575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259\": rpc error: code = NotFound desc = could not find container \"575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259\": container with ID starting with 575dfa27cce1390f6ccf90b7634631d13fba0e3dfcf0b270df198c0b3403f259 not found: ID does not exist"
Jul 17 06:31:19.249423 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:19.249401    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3b164bbd-openshift-default-8cbd7f987-fv9k8"]
Jul 17 06:31:20.079169 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:20.079136    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a936723f-8eca-467c-a3c4-5e868501e853" path="/var/lib/kubelet/pods/a936723f-8eca-467c-a3c4-5e868501e853/volumes"
Jul 17 06:31:30.190289 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190252    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"]
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190659    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190670    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190679    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerName="revoke-keys"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190685    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerName="revoke-keys"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190693    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerName="revoke-keys"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190698    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerName="revoke-keys"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190708    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerName="revoke-keys"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190713    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerName="revoke-keys"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190721    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerName="revoke-keys"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190726    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerName="revoke-keys"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190735    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerName="revoke-keys"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190740    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerName="revoke-keys"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190758    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerName="revoke-keys"
Jul 17 06:31:30.190760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190762    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerName="revoke-keys"
Jul 17 06:31:30.191216 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190815    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerName="revoke-keys"
Jul 17 06:31:30.191216 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190824    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy"
Jul 17 06:31:30.191216 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190830    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy"
Jul 17 06:31:30.191216 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190837    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerName="revoke-keys"
Jul 17 06:31:30.191216 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190842    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerName="revoke-keys"
Jul 17 06:31:30.191216 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.190849    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerName="revoke-keys"
Jul 17 06:31:30.193902 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.193886    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"
Jul 17 06:31:30.196984 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.196967    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:31:30.215351 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.215326    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"]
Jul 17 06:31:30.298912 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.298880    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/209f9bc3-04d9-4048-bb07-baa05b4dc9db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf\" (UID: \"209f9bc3-04d9-4048-bb07-baa05b4dc9db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"
Jul 17 06:31:30.399720 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.399698    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/209f9bc3-04d9-4048-bb07-baa05b4dc9db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf\" (UID: \"209f9bc3-04d9-4048-bb07-baa05b4dc9db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"
Jul 17 06:31:30.400330 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.400311    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/209f9bc3-04d9-4048-bb07-baa05b4dc9db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf\" (UID: \"209f9bc3-04d9-4048-bb07-baa05b4dc9db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"
Jul 17 06:31:30.504517 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.504422    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"
Jul 17 06:31:30.667260 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:30.667197    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"]
Jul 17 06:31:30.671119 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:31:30.671088    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209f9bc3_04d9_4048_bb07_baa05b4dc9db.slice/crio-450d9c12514e98307fc25f1cc0a6020be3f8e014f713d7c308e3a4526c54822e WatchSource:0}: Error finding container 450d9c12514e98307fc25f1cc0a6020be3f8e014f713d7c308e3a4526c54822e: Status 404 returned error can't find the container with id 450d9c12514e98307fc25f1cc0a6020be3f8e014f713d7c308e3a4526c54822e
Jul 17 06:31:31.285284 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:31.285259    2558 generic.go:356] "Generic (PLEG): container finished" podID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerID="19c7f965e7a709ec26095aac644848e51073fa9e2ee2211feef56a9caf517367" exitCode=6
Jul 17 06:31:31.285615 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:31.285348    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf" event={"ID":"209f9bc3-04d9-4048-bb07-baa05b4dc9db","Type":"ContainerDied","Data":"19c7f965e7a709ec26095aac644848e51073fa9e2ee2211feef56a9caf517367"}
Jul 17 06:31:31.285615 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:31.285410    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf" event={"ID":"209f9bc3-04d9-4048-bb07-baa05b4dc9db","Type":"ContainerStarted","Data":"450d9c12514e98307fc25f1cc0a6020be3f8e014f713d7c308e3a4526c54822e"}
Jul 17 06:31:31.285615 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:31.285523    2558 scope.go:117] "RemoveContainer" containerID="19c7f965e7a709ec26095aac644848e51073fa9e2ee2211feef56a9caf517367"
Jul 17 06:31:32.291774 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:32.291685    2558 generic.go:356] "Generic (PLEG): container finished" podID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerID="f7262bc12079c3ef59fab7ce35bdfa440a6f268472b793d4f7ae7b53e56c4027" exitCode=6
Jul 17 06:31:32.292206 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:32.291774    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf" event={"ID":"209f9bc3-04d9-4048-bb07-baa05b4dc9db","Type":"ContainerDied","Data":"f7262bc12079c3ef59fab7ce35bdfa440a6f268472b793d4f7ae7b53e56c4027"}
Jul 17 06:31:32.292206 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:32.291828    2558 scope.go:117] "RemoveContainer" containerID="19c7f965e7a709ec26095aac644848e51073fa9e2ee2211feef56a9caf517367"
Jul 17 06:31:32.292206 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:32.292001    2558 scope.go:117] "RemoveContainer" containerID="f7262bc12079c3ef59fab7ce35bdfa440a6f268472b793d4f7ae7b53e56c4027"
Jul 17 06:31:32.292360 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:31:32.292268    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf_odh-ai-gateway-infra(209f9bc3-04d9-4048-bb07-baa05b4dc9db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db"
Jul 17 06:31:33.298435 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:33.298408    2558 scope.go:117] "RemoveContainer" containerID="f7262bc12079c3ef59fab7ce35bdfa440a6f268472b793d4f7ae7b53e56c4027"
Jul 17 06:31:33.298801 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:31:33.298602    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf_odh-ai-gateway-infra(209f9bc3-04d9-4048-bb07-baa05b4dc9db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db"
Jul 17 06:31:44.167249 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.167218    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"]
Jul 17 06:31:44.167678 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.167644    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy"
Jul 17 06:31:44.167678 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.167657    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a936723f-8eca-467c-a3c4-5e868501e853" containerName="istio-proxy"
Jul 17 06:31:44.167752 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.167728    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="54d83c85-0c9c-415c-9a2b-7ee7c436cada" containerName="revoke-keys"
Jul 17 06:31:44.167752 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.167740    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="af3ac34a-b8f4-44ff-86d8-8d6ec5cc8668" containerName="revoke-keys"
Jul 17 06:31:44.172034 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.172017    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"
Jul 17 06:31:44.179839 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.179807    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"]
Jul 17 06:31:44.317983 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.317958    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64db4986-39a3-4835-9e7a-887a18951cfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-f9mts\" (UID: \"64db4986-39a3-4835-9e7a-887a18951cfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"
Jul 17 06:31:44.418487 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.418412    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64db4986-39a3-4835-9e7a-887a18951cfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-f9mts\" (UID: \"64db4986-39a3-4835-9e7a-887a18951cfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"
Jul 17 06:31:44.418984 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.418967    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64db4986-39a3-4835-9e7a-887a18951cfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-a9646996-f9mts\" (UID: \"64db4986-39a3-4835-9e7a-887a18951cfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"
Jul 17 06:31:44.483784 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.483760    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"
Jul 17 06:31:44.619831 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:44.619805    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"]
Jul 17 06:31:44.621403 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:31:44.621365    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64db4986_39a3_4835_9e7a_887a18951cfc.slice/crio-b1b7223abf042d46658bc009e95bec024f2096f799894ae706f77a90d8b96870 WatchSource:0}: Error finding container b1b7223abf042d46658bc009e95bec024f2096f799894ae706f77a90d8b96870: Status 404 returned error can't find the container with id b1b7223abf042d46658bc009e95bec024f2096f799894ae706f77a90d8b96870
Jul 17 06:31:45.073034 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:45.073008    2558 scope.go:117] "RemoveContainer" containerID="f7262bc12079c3ef59fab7ce35bdfa440a6f268472b793d4f7ae7b53e56c4027"
Jul 17 06:31:45.363297 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:45.363268    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf" event={"ID":"209f9bc3-04d9-4048-bb07-baa05b4dc9db","Type":"ContainerStarted","Data":"6f4806b420dda21176ff6e037d7192f6b88344adddeb47bbc6326e37059bb593"}
Jul 17 06:31:45.364774 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:45.364731    2558 generic.go:356] "Generic (PLEG): container finished" podID="64db4986-39a3-4835-9e7a-887a18951cfc" containerID="ce0f480508f8b019b450af0caf29b95a567fe20b17b74238d990f1442ef267a2" exitCode=6
Jul 17 06:31:45.364774 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:45.364770    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts" event={"ID":"64db4986-39a3-4835-9e7a-887a18951cfc","Type":"ContainerDied","Data":"ce0f480508f8b019b450af0caf29b95a567fe20b17b74238d990f1442ef267a2"}
Jul 17 06:31:45.364919 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:45.364804    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts" event={"ID":"64db4986-39a3-4835-9e7a-887a18951cfc","Type":"ContainerStarted","Data":"b1b7223abf042d46658bc009e95bec024f2096f799894ae706f77a90d8b96870"}
Jul 17 06:31:45.364919 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:45.364913    2558 scope.go:117] "RemoveContainer" containerID="ce0f480508f8b019b450af0caf29b95a567fe20b17b74238d990f1442ef267a2"
Jul 17 06:31:45.379449 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:45.379406    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf" podStartSLOduration=15.379394408 podStartE2EDuration="15.379394408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:31:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:31:45.37766856 +0000 UTC m=+3027.819607168" watchObservedRunningTime="2026-07-17 06:31:45.379394408 +0000 UTC m=+3027.821332984"
Jul 17 06:31:46.371452 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:46.371420    2558 generic.go:356] "Generic (PLEG): container finished" podID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerID="6f4806b420dda21176ff6e037d7192f6b88344adddeb47bbc6326e37059bb593" exitCode=6
Jul 17 06:31:46.371888 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:46.371493    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf" event={"ID":"209f9bc3-04d9-4048-bb07-baa05b4dc9db","Type":"ContainerDied","Data":"6f4806b420dda21176ff6e037d7192f6b88344adddeb47bbc6326e37059bb593"}
Jul 17 06:31:46.371888 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:46.371562    2558 scope.go:117] "RemoveContainer" containerID="f7262bc12079c3ef59fab7ce35bdfa440a6f268472b793d4f7ae7b53e56c4027"
Jul 17 06:31:46.371888 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:46.371799    2558 scope.go:117] "RemoveContainer" containerID="6f4806b420dda21176ff6e037d7192f6b88344adddeb47bbc6326e37059bb593"
Jul 17 06:31:46.372039 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:31:46.372020    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf_odh-ai-gateway-infra(209f9bc3-04d9-4048-bb07-baa05b4dc9db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db"
Jul 17 06:31:46.373601 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:46.373576    2558 generic.go:356] "Generic (PLEG): container finished" podID="64db4986-39a3-4835-9e7a-887a18951cfc" containerID="d867786a2aab095df069c1c44ee85ec4e1327895558d574e234e5a7888c6e198" exitCode=6
Jul 17 06:31:46.373703 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:46.373638    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts" event={"ID":"64db4986-39a3-4835-9e7a-887a18951cfc","Type":"ContainerDied","Data":"d867786a2aab095df069c1c44ee85ec4e1327895558d574e234e5a7888c6e198"}
Jul 17 06:31:46.373828 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:46.373816    2558 scope.go:117] "RemoveContainer" containerID="d867786a2aab095df069c1c44ee85ec4e1327895558d574e234e5a7888c6e198"
Jul 17 06:31:46.374010 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:31:46.373995    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-f9mts_odh-ai-gateway-infra(64db4986-39a3-4835-9e7a-887a18951cfc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts" podUID="64db4986-39a3-4835-9e7a-887a18951cfc"
Jul 17 06:31:46.382533 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:46.382517    2558 scope.go:117] "RemoveContainer" containerID="ce0f480508f8b019b450af0caf29b95a567fe20b17b74238d990f1442ef267a2"
Jul 17 06:31:46.395851 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:46.395831    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"]
Jul 17 06:31:47.381137 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:47.381109    2558 scope.go:117] "RemoveContainer" containerID="d867786a2aab095df069c1c44ee85ec4e1327895558d574e234e5a7888c6e198"
Jul 17 06:31:47.381567 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:31:47.381316    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-f9mts_odh-ai-gateway-infra(64db4986-39a3-4835-9e7a-887a18951cfc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts" podUID="64db4986-39a3-4835-9e7a-887a18951cfc"
Jul 17 06:31:47.527334 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:47.527308    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"
Jul 17 06:31:47.646452 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:47.646384    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/209f9bc3-04d9-4048-bb07-baa05b4dc9db-maas-api-service-ca\") pod \"209f9bc3-04d9-4048-bb07-baa05b4dc9db\" (UID: \"209f9bc3-04d9-4048-bb07-baa05b4dc9db\") "
Jul 17 06:31:47.646747 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:47.646725    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/209f9bc3-04d9-4048-bb07-baa05b4dc9db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "209f9bc3-04d9-4048-bb07-baa05b4dc9db" (UID: "209f9bc3-04d9-4048-bb07-baa05b4dc9db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:31:47.747828 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:47.747799    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/209f9bc3-04d9-4048-bb07-baa05b4dc9db-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:31:48.389151 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:48.389070    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"
Jul 17 06:31:48.389151 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:48.389100    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf" event={"ID":"209f9bc3-04d9-4048-bb07-baa05b4dc9db","Type":"ContainerDied","Data":"450d9c12514e98307fc25f1cc0a6020be3f8e014f713d7c308e3a4526c54822e"}
Jul 17 06:31:48.389618 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:48.389160    2558 scope.go:117] "RemoveContainer" containerID="6f4806b420dda21176ff6e037d7192f6b88344adddeb47bbc6326e37059bb593"
Jul 17 06:31:48.411331 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:48.411308    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"]
Jul 17 06:31:48.415162 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:48.415141    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-6j7hf"]
Jul 17 06:31:50.078132 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:31:50.078099    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" path="/var/lib/kubelet/pods/209f9bc3-04d9-4048-bb07-baa05b4dc9db/volumes"
Jul 17 06:32:01.072405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:01.072376    2558 scope.go:117] "RemoveContainer" containerID="d867786a2aab095df069c1c44ee85ec4e1327895558d574e234e5a7888c6e198"
Jul 17 06:32:01.463363 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:01.463329    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts" event={"ID":"64db4986-39a3-4835-9e7a-887a18951cfc","Type":"ContainerStarted","Data":"33f8e59f950a93e47389efcfa7b021336c276fcf6358d724d35ea33bcc8fdd31"}
Jul 17 06:32:01.487864 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:01.487803    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts" podStartSLOduration=17.487784195 podStartE2EDuration="17.487784195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:31:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:32:01.487239636 +0000 UTC m=+3043.929178225" watchObservedRunningTime="2026-07-17 06:32:01.487784195 +0000 UTC m=+3043.929722770"
Jul 17 06:32:02.470801 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.470764    2558 generic.go:356] "Generic (PLEG): container finished" podID="64db4986-39a3-4835-9e7a-887a18951cfc" containerID="33f8e59f950a93e47389efcfa7b021336c276fcf6358d724d35ea33bcc8fdd31" exitCode=6
Jul 17 06:32:02.471271 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.470843    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts" event={"ID":"64db4986-39a3-4835-9e7a-887a18951cfc","Type":"ContainerDied","Data":"33f8e59f950a93e47389efcfa7b021336c276fcf6358d724d35ea33bcc8fdd31"}
Jul 17 06:32:02.471271 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.470920    2558 scope.go:117] "RemoveContainer" containerID="d867786a2aab095df069c1c44ee85ec4e1327895558d574e234e5a7888c6e198"
Jul 17 06:32:02.471271 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.471252    2558 scope.go:117] "RemoveContainer" containerID="33f8e59f950a93e47389efcfa7b021336c276fcf6358d724d35ea33bcc8fdd31"
Jul 17 06:32:02.471465 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:32:02.471446    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-a9646996-f9mts_odh-ai-gateway-infra(64db4986-39a3-4835-9e7a-887a18951cfc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts" podUID="64db4986-39a3-4835-9e7a-887a18951cfc"
Jul 17 06:32:02.499956 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.499928    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"]
Jul 17 06:32:02.603106 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.603065    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"]
Jul 17 06:32:02.603540 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.603527    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerName="revoke-keys"
Jul 17 06:32:02.603585 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.603541    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerName="revoke-keys"
Jul 17 06:32:02.603585 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.603553    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerName="revoke-keys"
Jul 17 06:32:02.603585 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.603559    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerName="revoke-keys"
Jul 17 06:32:02.603585 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.603577    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerName="revoke-keys"
Jul 17 06:32:02.603585 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.603583    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerName="revoke-keys"
Jul 17 06:32:02.603754 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.603646    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerName="revoke-keys"
Jul 17 06:32:02.603754 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.603654    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerName="revoke-keys"
Jul 17 06:32:02.607831 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.607813    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"
Jul 17 06:32:02.614427 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.614398    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"]
Jul 17 06:32:02.672389 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.672342    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/427b49fe-c184-4cbc-b40d-2a0c4754c387-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-gzqqh\" (UID: \"427b49fe-c184-4cbc-b40d-2a0c4754c387\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"
Jul 17 06:32:02.773414 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.773313    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/427b49fe-c184-4cbc-b40d-2a0c4754c387-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-gzqqh\" (UID: \"427b49fe-c184-4cbc-b40d-2a0c4754c387\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"
Jul 17 06:32:02.773976 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.773956    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/427b49fe-c184-4cbc-b40d-2a0c4754c387-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-572ab047-gzqqh\" (UID: \"427b49fe-c184-4cbc-b40d-2a0c4754c387\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"
Jul 17 06:32:02.920220 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:02.920186    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"
Jul 17 06:32:03.075381 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:03.075295    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"]
Jul 17 06:32:03.077786 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:32:03.077757    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod427b49fe_c184_4cbc_b40d_2a0c4754c387.slice/crio-a51cfe14d467c8ab2c68f65672553150ff7f0c39565b7e58cc160080d9d560f4 WatchSource:0}: Error finding container a51cfe14d467c8ab2c68f65672553150ff7f0c39565b7e58cc160080d9d560f4: Status 404 returned error can't find the container with id a51cfe14d467c8ab2c68f65672553150ff7f0c39565b7e58cc160080d9d560f4
Jul 17 06:32:03.479621 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:03.479579    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh" event={"ID":"427b49fe-c184-4cbc-b40d-2a0c4754c387","Type":"ContainerStarted","Data":"54497b4b0e54d9f60e3ef7b27f2b945cd116cad664626407129e3a5704506a91"}
Jul 17 06:32:03.479941 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:03.479634    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh" event={"ID":"427b49fe-c184-4cbc-b40d-2a0c4754c387","Type":"ContainerStarted","Data":"a51cfe14d467c8ab2c68f65672553150ff7f0c39565b7e58cc160080d9d560f4"}
Jul 17 06:32:03.520820 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:03.520762    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh" podStartSLOduration=1.520743883 podStartE2EDuration="1.520743883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:32:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:32:03.519799865 +0000 UTC m=+3045.961738452" watchObservedRunningTime="2026-07-17 06:32:03.520743883 +0000 UTC m=+3045.962682458"
Jul 17 06:32:03.622428 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:03.622402    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"
Jul 17 06:32:03.680075 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:03.680036    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64db4986-39a3-4835-9e7a-887a18951cfc-maas-api-service-ca\") pod \"64db4986-39a3-4835-9e7a-887a18951cfc\" (UID: \"64db4986-39a3-4835-9e7a-887a18951cfc\") "
Jul 17 06:32:03.680456 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:03.680430    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64db4986-39a3-4835-9e7a-887a18951cfc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "64db4986-39a3-4835-9e7a-887a18951cfc" (UID: "64db4986-39a3-4835-9e7a-887a18951cfc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:32:03.781257 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:03.781206    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64db4986-39a3-4835-9e7a-887a18951cfc-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:32:04.486630 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:04.486593    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"
Jul 17 06:32:04.486630 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:04.486613    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts" event={"ID":"64db4986-39a3-4835-9e7a-887a18951cfc","Type":"ContainerDied","Data":"b1b7223abf042d46658bc009e95bec024f2096f799894ae706f77a90d8b96870"}
Jul 17 06:32:04.487189 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:04.486688    2558 scope.go:117] "RemoveContainer" containerID="33f8e59f950a93e47389efcfa7b021336c276fcf6358d724d35ea33bcc8fdd31"
Jul 17 06:32:04.488377 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:04.488348    2558 generic.go:356] "Generic (PLEG): container finished" podID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerID="54497b4b0e54d9f60e3ef7b27f2b945cd116cad664626407129e3a5704506a91" exitCode=6
Jul 17 06:32:04.488547 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:04.488444    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh" event={"ID":"427b49fe-c184-4cbc-b40d-2a0c4754c387","Type":"ContainerDied","Data":"54497b4b0e54d9f60e3ef7b27f2b945cd116cad664626407129e3a5704506a91"}
Jul 17 06:32:04.488645 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:04.488629    2558 scope.go:117] "RemoveContainer" containerID="54497b4b0e54d9f60e3ef7b27f2b945cd116cad664626407129e3a5704506a91"
Jul 17 06:32:04.559782 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:04.559754    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"]
Jul 17 06:32:04.570894 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:04.570864    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-a9646996-f9mts"]
Jul 17 06:32:05.497146 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:05.497069    2558 generic.go:356] "Generic (PLEG): container finished" podID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerID="24fb920b9001ee029ea64b12677ffe25eba309e1c8f61685fc47c58bb2f83fce" exitCode=6
Jul 17 06:32:05.497146 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:05.497131    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh" event={"ID":"427b49fe-c184-4cbc-b40d-2a0c4754c387","Type":"ContainerDied","Data":"24fb920b9001ee029ea64b12677ffe25eba309e1c8f61685fc47c58bb2f83fce"}
Jul 17 06:32:05.497656 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:05.497181    2558 scope.go:117] "RemoveContainer" containerID="54497b4b0e54d9f60e3ef7b27f2b945cd116cad664626407129e3a5704506a91"
Jul 17 06:32:05.497656 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:05.497330    2558 scope.go:117] "RemoveContainer" containerID="24fb920b9001ee029ea64b12677ffe25eba309e1c8f61685fc47c58bb2f83fce"
Jul 17 06:32:05.497656 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:32:05.497600    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-gzqqh_odh-ai-gateway-infra(427b49fe-c184-4cbc-b40d-2a0c4754c387)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387"
Jul 17 06:32:06.078760 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.078726    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64db4986-39a3-4835-9e7a-887a18951cfc" path="/var/lib/kubelet/pods/64db4986-39a3-4835-9e7a-887a18951cfc/volumes"
Jul 17 06:32:06.280397 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.280330    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"]
Jul 17 06:32:06.280981 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.280949    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64db4986-39a3-4835-9e7a-887a18951cfc" containerName="revoke-keys"
Jul 17 06:32:06.280981 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.280982    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="64db4986-39a3-4835-9e7a-887a18951cfc" containerName="revoke-keys"
Jul 17 06:32:06.281073 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.280995    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64db4986-39a3-4835-9e7a-887a18951cfc" containerName="revoke-keys"
Jul 17 06:32:06.281073 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.281002    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="64db4986-39a3-4835-9e7a-887a18951cfc" containerName="revoke-keys"
Jul 17 06:32:06.281147 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.281075    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="64db4986-39a3-4835-9e7a-887a18951cfc" containerName="revoke-keys"
Jul 17 06:32:06.281147 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.281085    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="64db4986-39a3-4835-9e7a-887a18951cfc" containerName="revoke-keys"
Jul 17 06:32:06.281147 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.281094    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="209f9bc3-04d9-4048-bb07-baa05b4dc9db" containerName="revoke-keys"
Jul 17 06:32:06.286024 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.285973    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"
Jul 17 06:32:06.299170 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.299133    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23032bf-d45a-4fdf-80fa-17f2e07159f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj\" (UID: \"d23032bf-d45a-4fdf-80fa-17f2e07159f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"
Jul 17 06:32:06.301364 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.301339    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"]
Jul 17 06:32:06.399765 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.399728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23032bf-d45a-4fdf-80fa-17f2e07159f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj\" (UID: \"d23032bf-d45a-4fdf-80fa-17f2e07159f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"
Jul 17 06:32:06.400403 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.400383    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23032bf-d45a-4fdf-80fa-17f2e07159f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj\" (UID: \"d23032bf-d45a-4fdf-80fa-17f2e07159f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"
Jul 17 06:32:06.505210 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.505179    2558 scope.go:117] "RemoveContainer" containerID="24fb920b9001ee029ea64b12677ffe25eba309e1c8f61685fc47c58bb2f83fce"
Jul 17 06:32:06.505633 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:32:06.505490    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-gzqqh_odh-ai-gateway-infra(427b49fe-c184-4cbc-b40d-2a0c4754c387)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387"
Jul 17 06:32:06.598877 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.598831    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"
Jul 17 06:32:06.753926 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:06.753903    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"]
Jul 17 06:32:06.755154 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:32:06.755125    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23032bf_d45a_4fdf_80fa_17f2e07159f0.slice/crio-baf219610fe8defe59de4bd4b3124678d0deb4b11407b31a852593d673645a54 WatchSource:0}: Error finding container baf219610fe8defe59de4bd4b3124678d0deb4b11407b31a852593d673645a54: Status 404 returned error can't find the container with id baf219610fe8defe59de4bd4b3124678d0deb4b11407b31a852593d673645a54
Jul 17 06:32:07.512024 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:07.511980    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj" event={"ID":"d23032bf-d45a-4fdf-80fa-17f2e07159f0","Type":"ContainerStarted","Data":"37ac983fc22a5be433851d470f1bc6da2b2dca5a87479693de841f12ac3af70c"}
Jul 17 06:32:07.512401 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:07.512045    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj" event={"ID":"d23032bf-d45a-4fdf-80fa-17f2e07159f0","Type":"ContainerStarted","Data":"baf219610fe8defe59de4bd4b3124678d0deb4b11407b31a852593d673645a54"}
Jul 17 06:32:07.548229 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:07.548184    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj" podStartSLOduration=1.5481683689999999 podStartE2EDuration="1.548168369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:32:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:32:07.546143765 +0000 UTC m=+3049.988082340" watchObservedRunningTime="2026-07-17 06:32:07.548168369 +0000 UTC m=+3049.990106944"
Jul 17 06:32:08.519972 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:08.519875    2558 generic.go:356] "Generic (PLEG): container finished" podID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerID="37ac983fc22a5be433851d470f1bc6da2b2dca5a87479693de841f12ac3af70c" exitCode=6
Jul 17 06:32:08.520338 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:08.519963    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj" event={"ID":"d23032bf-d45a-4fdf-80fa-17f2e07159f0","Type":"ContainerDied","Data":"37ac983fc22a5be433851d470f1bc6da2b2dca5a87479693de841f12ac3af70c"}
Jul 17 06:32:08.520338 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:08.520163    2558 scope.go:117] "RemoveContainer" containerID="37ac983fc22a5be433851d470f1bc6da2b2dca5a87479693de841f12ac3af70c"
Jul 17 06:32:09.528234 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:09.528132    2558 generic.go:356] "Generic (PLEG): container finished" podID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerID="92873476e1a8cb99747d4f3ce3be4f5f126fe67198279beb3cc8a718cc9146c4" exitCode=6
Jul 17 06:32:09.528234 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:09.528207    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj" event={"ID":"d23032bf-d45a-4fdf-80fa-17f2e07159f0","Type":"ContainerDied","Data":"92873476e1a8cb99747d4f3ce3be4f5f126fe67198279beb3cc8a718cc9146c4"}
Jul 17 06:32:09.528780 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:09.528265    2558 scope.go:117] "RemoveContainer" containerID="37ac983fc22a5be433851d470f1bc6da2b2dca5a87479693de841f12ac3af70c"
Jul 17 06:32:09.528780 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:09.528512    2558 scope.go:117] "RemoveContainer" containerID="92873476e1a8cb99747d4f3ce3be4f5f126fe67198279beb3cc8a718cc9146c4"
Jul 17 06:32:09.528780 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:32:09.528759    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj_odh-ai-gateway-infra(d23032bf-d45a-4fdf-80fa-17f2e07159f0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0"
Jul 17 06:32:10.534906 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.534873    2558 scope.go:117] "RemoveContainer" containerID="92873476e1a8cb99747d4f3ce3be4f5f126fe67198279beb3cc8a718cc9146c4"
Jul 17 06:32:10.535327 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:32:10.535165    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj_odh-ai-gateway-infra(d23032bf-d45a-4fdf-80fa-17f2e07159f0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0"
Jul 17 06:32:10.870945 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.870909    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"]
Jul 17 06:32:10.871372 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.871359    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64db4986-39a3-4835-9e7a-887a18951cfc" containerName="revoke-keys"
Jul 17 06:32:10.871413 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.871375    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="64db4986-39a3-4835-9e7a-887a18951cfc" containerName="revoke-keys"
Jul 17 06:32:10.871466 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.871457    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="64db4986-39a3-4835-9e7a-887a18951cfc" containerName="revoke-keys"
Jul 17 06:32:10.875703 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.875682    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:10.879664 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.879644    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e77d3d-openshift-default-dockercfg-h98kg\""
Jul 17 06:32:10.902561 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.902541    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"]
Jul 17 06:32:10.941041 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.941012    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-socket\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:10.941215 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.941047    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-podinfo\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:10.941215 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.941071    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-data\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:10.941215 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.941096    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f83520fc-8187-4700-93c3-47f4adbfe4ab-istiod-ca-cert\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:10.941215 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.941137    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-envoy\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:10.941383 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.941224    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-credential-socket\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:10.941383 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.941271    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-token\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:10.941383 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.941301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlnr7\" (UniqueName: \"kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-kube-api-access-jlnr7\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:10.941383 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:10.941347    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-certs\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.042540 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.042507    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-token\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.042715 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.042589    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlnr7\" (UniqueName: \"kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-kube-api-access-jlnr7\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.042715 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.042626    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-certs\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.042715 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.042675    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-socket\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.042914 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.042844    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-podinfo\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.042914 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.042895    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-data\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.043013 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.042926    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f83520fc-8187-4700-93c3-47f4adbfe4ab-istiod-ca-cert\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.043013 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.042961    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-envoy\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.043190 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.043027    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-credential-socket\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.043190 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.043089    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-socket\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.043322 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.043295    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-data\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.043424 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.043407    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-certs\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.043517 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.043467    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-credential-socket\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.043713 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.043691    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f83520fc-8187-4700-93c3-47f4adbfe4ab-istiod-ca-cert\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.045406 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.045389    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-envoy\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.045669 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.045635    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-podinfo\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.051653 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.051633    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlnr7\" (UniqueName: \"kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-kube-api-access-jlnr7\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.052188 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.052166    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-token\") pod \"e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") " pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.186640 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.186538    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:11.356697 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.356663    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"]
Jul 17 06:32:11.356824 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:32:11.356709    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83520fc_8187_4700_93c3_47f4adbfe4ab.slice/crio-c8c5379e3684398ca8a678f8e8cc135985de98be65092c169e72bd6a35e13ce0 WatchSource:0}: Error finding container c8c5379e3684398ca8a678f8e8cc135985de98be65092c169e72bd6a35e13ce0: Status 404 returned error can't find the container with id c8c5379e3684398ca8a678f8e8cc135985de98be65092c169e72bd6a35e13ce0
Jul 17 06:32:11.358965 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.358932    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:11.359102 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.359012    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:11.359102 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.359064    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:11.542738 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.542638    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" event={"ID":"f83520fc-8187-4700-93c3-47f4adbfe4ab","Type":"ContainerStarted","Data":"7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2"}
Jul 17 06:32:11.542738 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.542686    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" event={"ID":"f83520fc-8187-4700-93c3-47f4adbfe4ab","Type":"ContainerStarted","Data":"c8c5379e3684398ca8a678f8e8cc135985de98be65092c169e72bd6a35e13ce0"}
Jul 17 06:32:11.568094 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:11.568024    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" podStartSLOduration=1.568004081 podStartE2EDuration="1.568004081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:32:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:32:11.564143303 +0000 UTC m=+3054.006081924" watchObservedRunningTime="2026-07-17 06:32:11.568004081 +0000 UTC m=+3054.009942653"
Jul 17 06:32:12.187066 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:12.187025    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:12.191989 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:12.191965    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:12.548175 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:12.548101    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:12.549110 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:12.549091    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:18.618194 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.618163    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"]
Jul 17 06:32:18.623292 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.623158    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.626209 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.626185    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-142438-openshift-default-dockercfg-s5fdl\""
Jul 17 06:32:18.636829 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.636804    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"]
Jul 17 06:32:18.720607 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.720506    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-envoy\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.720607 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.720562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-podinfo\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.720837 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.720610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-certs\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.720837 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.720633    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-data\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.720837 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.720695    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-token\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.720837 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.720738    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-socket\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.720837 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.720774    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-credential-socket\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.720837 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.720815    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3022e42b-791d-4281-a4f1-2bb70558bfeb-istiod-ca-cert\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.721181 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.720880    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr52z\" (UniqueName: \"kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-kube-api-access-rr52z\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.822244 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.822206    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-envoy\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.822244 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.822259    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-podinfo\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.822561 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.822296    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-certs\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.822561 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.822316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-data\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.822561 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.822362    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-token\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.822561 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.822395    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-socket\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.822561 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.822424    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-credential-socket\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.822561 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.822460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3022e42b-791d-4281-a4f1-2bb70558bfeb-istiod-ca-cert\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.822561 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.822548    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr52z\" (UniqueName: \"kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-kube-api-access-rr52z\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.822973 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.822778    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-certs\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.823645 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.823620    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3022e42b-791d-4281-a4f1-2bb70558bfeb-istiod-ca-cert\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.825094 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.825069    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-envoy\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.825340 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.825317    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-data\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.825531 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.825394    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-podinfo\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.825693 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.825648    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-socket\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.825870 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.825846    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-credential-socket\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.832284 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.832252    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-token\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.832373 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.832352    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr52z\" (UniqueName: \"kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-kube-api-access-rr52z\") pod \"e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") " pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:18.934699 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:18.934667    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:19.080265 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:19.080234    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"]
Jul 17 06:32:19.081140 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:32:19.081114    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3022e42b_791d_4281_a4f1_2bb70558bfeb.slice/crio-9878588321e6bb2284e63b107a785e4c5e7f5ec975380e77f07df0edcee2d26f WatchSource:0}: Error finding container 9878588321e6bb2284e63b107a785e4c5e7f5ec975380e77f07df0edcee2d26f: Status 404 returned error can't find the container with id 9878588321e6bb2284e63b107a785e4c5e7f5ec975380e77f07df0edcee2d26f
Jul 17 06:32:19.083234 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:19.083206    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:19.083305 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:19.083287    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:19.083345 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:19.083325    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:19.593173 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:19.593136    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" event={"ID":"3022e42b-791d-4281-a4f1-2bb70558bfeb","Type":"ContainerStarted","Data":"37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e"}
Jul 17 06:32:19.593393 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:19.593202    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" event={"ID":"3022e42b-791d-4281-a4f1-2bb70558bfeb","Type":"ContainerStarted","Data":"9878588321e6bb2284e63b107a785e4c5e7f5ec975380e77f07df0edcee2d26f"}
Jul 17 06:32:19.616900 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:19.616845    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" podStartSLOduration=1.6168256319999998 podStartE2EDuration="1.616825632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:32:19.615011105 +0000 UTC m=+3062.056949708" watchObservedRunningTime="2026-07-17 06:32:19.616825632 +0000 UTC m=+3062.058764208"
Jul 17 06:32:19.935029 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:19.934942    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:20.940559 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:20.940531    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:21.072419 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:21.072387    2558 scope.go:117] "RemoveContainer" containerID="24fb920b9001ee029ea64b12677ffe25eba309e1c8f61685fc47c58bb2f83fce"
Jul 17 06:32:21.606273 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:21.606233    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh" event={"ID":"427b49fe-c184-4cbc-b40d-2a0c4754c387","Type":"ContainerStarted","Data":"cffe6a3ff86075e27529a2125601ecf5ae9c947af590c390e0c4a03ef1ec2a4d"}
Jul 17 06:32:21.606512 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:21.606463    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:21.607443 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:21.607424    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:22.616456 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:22.616413    2558 generic.go:356] "Generic (PLEG): container finished" podID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerID="cffe6a3ff86075e27529a2125601ecf5ae9c947af590c390e0c4a03ef1ec2a4d" exitCode=6
Jul 17 06:32:22.616939 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:22.616519    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh" event={"ID":"427b49fe-c184-4cbc-b40d-2a0c4754c387","Type":"ContainerDied","Data":"cffe6a3ff86075e27529a2125601ecf5ae9c947af590c390e0c4a03ef1ec2a4d"}
Jul 17 06:32:22.616939 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:22.616613    2558 scope.go:117] "RemoveContainer" containerID="24fb920b9001ee029ea64b12677ffe25eba309e1c8f61685fc47c58bb2f83fce"
Jul 17 06:32:22.617115 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:22.617097    2558 scope.go:117] "RemoveContainer" containerID="cffe6a3ff86075e27529a2125601ecf5ae9c947af590c390e0c4a03ef1ec2a4d"
Jul 17 06:32:22.617303 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:32:22.617283    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-572ab047-gzqqh_odh-ai-gateway-infra(427b49fe-c184-4cbc-b40d-2a0c4754c387)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387"
Jul 17 06:32:22.646579 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:22.646540    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"]
Jul 17 06:32:23.816562 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:23.816536    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"
Jul 17 06:32:23.873678 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:23.873644    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/427b49fe-c184-4cbc-b40d-2a0c4754c387-maas-api-service-ca\") pod \"427b49fe-c184-4cbc-b40d-2a0c4754c387\" (UID: \"427b49fe-c184-4cbc-b40d-2a0c4754c387\") "
Jul 17 06:32:23.873980 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:23.873955    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/427b49fe-c184-4cbc-b40d-2a0c4754c387-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "427b49fe-c184-4cbc-b40d-2a0c4754c387" (UID: "427b49fe-c184-4cbc-b40d-2a0c4754c387"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:32:23.975264 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:23.975173    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/427b49fe-c184-4cbc-b40d-2a0c4754c387-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:32:24.072447 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:24.072422    2558 scope.go:117] "RemoveContainer" containerID="92873476e1a8cb99747d4f3ce3be4f5f126fe67198279beb3cc8a718cc9146c4"
Jul 17 06:32:24.637525 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:24.635377    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh" event={"ID":"427b49fe-c184-4cbc-b40d-2a0c4754c387","Type":"ContainerDied","Data":"a51cfe14d467c8ab2c68f65672553150ff7f0c39565b7e58cc160080d9d560f4"}
Jul 17 06:32:24.637525 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:24.635452    2558 scope.go:117] "RemoveContainer" containerID="cffe6a3ff86075e27529a2125601ecf5ae9c947af590c390e0c4a03ef1ec2a4d"
Jul 17 06:32:24.637525 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:24.635565    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"
Jul 17 06:32:24.644232 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:24.643735    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj" event={"ID":"d23032bf-d45a-4fdf-80fa-17f2e07159f0","Type":"ContainerStarted","Data":"545b2f7a5f233d522f50a9fd02f7124793779bf72635100bd8ee0fb08b3d40f3"}
Jul 17 06:32:24.665744 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:24.665721    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"]
Jul 17 06:32:24.669350 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:24.669332    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-572ab047-gzqqh"]
Jul 17 06:32:25.656263 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:25.656224    2558 generic.go:356] "Generic (PLEG): container finished" podID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerID="545b2f7a5f233d522f50a9fd02f7124793779bf72635100bd8ee0fb08b3d40f3" exitCode=6
Jul 17 06:32:25.656814 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:25.656304    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj" event={"ID":"d23032bf-d45a-4fdf-80fa-17f2e07159f0","Type":"ContainerDied","Data":"545b2f7a5f233d522f50a9fd02f7124793779bf72635100bd8ee0fb08b3d40f3"}
Jul 17 06:32:25.656814 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:25.656377    2558 scope.go:117] "RemoveContainer" containerID="92873476e1a8cb99747d4f3ce3be4f5f126fe67198279beb3cc8a718cc9146c4"
Jul 17 06:32:25.657016 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:25.656999    2558 scope.go:117] "RemoveContainer" containerID="545b2f7a5f233d522f50a9fd02f7124793779bf72635100bd8ee0fb08b3d40f3"
Jul 17 06:32:25.657283 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:32:25.657251    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj_odh-ai-gateway-infra(d23032bf-d45a-4fdf-80fa-17f2e07159f0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0"
Jul 17 06:32:25.719390 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:25.719364    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"]
Jul 17 06:32:26.078444 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:26.078361    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387" path="/var/lib/kubelet/pods/427b49fe-c184-4cbc-b40d-2a0c4754c387/volumes"
Jul 17 06:32:26.821892 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:26.821870    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"
Jul 17 06:32:26.902589 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:26.902556    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23032bf-d45a-4fdf-80fa-17f2e07159f0-maas-api-service-ca\") pod \"d23032bf-d45a-4fdf-80fa-17f2e07159f0\" (UID: \"d23032bf-d45a-4fdf-80fa-17f2e07159f0\") "
Jul 17 06:32:26.902874 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:26.902853    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d23032bf-d45a-4fdf-80fa-17f2e07159f0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d23032bf-d45a-4fdf-80fa-17f2e07159f0" (UID: "d23032bf-d45a-4fdf-80fa-17f2e07159f0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:32:26.903007 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:26.902980    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23032bf-d45a-4fdf-80fa-17f2e07159f0-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:32:27.670150 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:27.670122    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"
Jul 17 06:32:27.670341 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:27.670162    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj" event={"ID":"d23032bf-d45a-4fdf-80fa-17f2e07159f0","Type":"ContainerDied","Data":"baf219610fe8defe59de4bd4b3124678d0deb4b11407b31a852593d673645a54"}
Jul 17 06:32:27.670341 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:27.670222    2558 scope.go:117] "RemoveContainer" containerID="545b2f7a5f233d522f50a9fd02f7124793779bf72635100bd8ee0fb08b3d40f3"
Jul 17 06:32:27.694138 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:27.694066    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"]
Jul 17 06:32:27.697904 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:27.697882    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c567a84-tx8tj"]
Jul 17 06:32:28.080367 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:28.080288    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" path="/var/lib/kubelet/pods/d23032bf-d45a-4fdf-80fa-17f2e07159f0/volumes"
Jul 17 06:32:28.589707 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:28.589663    2558 patch_prober.go:28] interesting pod/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 06:32:28.589898 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:28.589738    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 06:32:30.688918 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:30.688884    2558 generic.go:356] "Generic (PLEG): container finished" podID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerID="7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2" exitCode=0
Jul 17 06:32:30.689331 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:30.688938    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" event={"ID":"f83520fc-8187-4700-93c3-47f4adbfe4ab","Type":"ContainerDied","Data":"7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2"}
Jul 17 06:32:30.689331 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:30.689274    2558 scope.go:117] "RemoveContainer" containerID="7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2"
Jul 17 06:32:30.690251 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:30.690219    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:30.690351 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:30.690306    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:30.690416 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:30.690356    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:31.695960 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:31.695927    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" event={"ID":"f83520fc-8187-4700-93c3-47f4adbfe4ab","Type":"ContainerStarted","Data":"d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5"}
Jul 17 06:32:32.187300 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:32.187259    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:32.187637 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:32.187611    2558 patch_prober.go:28] interesting pod/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Jul 17 06:32:32.187759 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:32.187668    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Jul 17 06:32:33.187413 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:33.187362    2558 patch_prober.go:28] interesting pod/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Jul 17 06:32:33.187812 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:33.187432    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Jul 17 06:32:34.187323 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:34.187279    2558 patch_prober.go:28] interesting pod/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Jul 17 06:32:34.187510 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:34.187347    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Jul 17 06:32:35.187913 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:35.187871    2558 patch_prober.go:28] interesting pod/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Jul 17 06:32:35.188309 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:35.187949    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Jul 17 06:32:36.191361 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:36.191333    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:36.191857 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:36.191834    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:36.192666 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:36.192645    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:32:37.648738 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:37.648680    2558 patch_prober.go:28] interesting pod/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 06:32:37.649268 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:37.648763    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 06:32:38.416842 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:32:38.416812    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3022e42b_791d_4281_a4f1_2bb70558bfeb.slice/crio-conmon-37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3022e42b_791d_4281_a4f1_2bb70558bfeb.slice/crio-37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 06:32:38.745177 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:38.745088    2558 generic.go:356] "Generic (PLEG): container finished" podID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerID="37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e" exitCode=0
Jul 17 06:32:38.745177 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:38.745161    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" event={"ID":"3022e42b-791d-4281-a4f1-2bb70558bfeb","Type":"ContainerDied","Data":"37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e"}
Jul 17 06:32:38.745710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:38.745682    2558 scope.go:117] "RemoveContainer" containerID="37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e"
Jul 17 06:32:38.746697 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:38.746671    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:38.746763 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:38.746740    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:38.746801 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:38.746770    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:32:39.755004 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:39.754944    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" event={"ID":"3022e42b-791d-4281-a4f1-2bb70558bfeb","Type":"ContainerStarted","Data":"e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e"}
Jul 17 06:32:39.935556 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:39.935518    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:39.936363 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:39.936321    2558 patch_prober.go:28] interesting pod/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Jul 17 06:32:39.936540 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:39.936415    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Jul 17 06:32:40.935900 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:40.935363    2558 patch_prober.go:28] interesting pod/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Jul 17 06:32:40.935900 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:40.935437    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Jul 17 06:32:41.935501 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:41.935429    2558 patch_prober.go:28] interesting pod/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Jul 17 06:32:41.935821 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:41.935787    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Jul 17 06:32:42.935452 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:42.935409    2558 patch_prober.go:28] interesting pod/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Jul 17 06:32:42.935977 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:42.935506    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Jul 17 06:32:43.938739 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:43.938707    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:43.939158 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:43.939139    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:32:43.939960 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:32:43.939939    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:34:08.596138 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:08.596106    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"]
Jul 17 06:34:08.597087 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:08.596329    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy" containerID="cri-o://d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5" gracePeriod=30
Jul 17 06:34:13.845781 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.845756    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:34:13.988431 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988349    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-token\") pod \"f83520fc-8187-4700-93c3-47f4adbfe4ab\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") "
Jul 17 06:34:13.988431 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988386    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-data\") pod \"f83520fc-8187-4700-93c3-47f4adbfe4ab\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") "
Jul 17 06:34:13.988431 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988413    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-envoy\") pod \"f83520fc-8187-4700-93c3-47f4adbfe4ab\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") "
Jul 17 06:34:13.988758 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988444    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-certs\") pod \"f83520fc-8187-4700-93c3-47f4adbfe4ab\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") "
Jul 17 06:34:13.988758 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988521    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-credential-socket\") pod \"f83520fc-8187-4700-93c3-47f4adbfe4ab\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") "
Jul 17 06:34:13.988758 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988550    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-socket\") pod \"f83520fc-8187-4700-93c3-47f4adbfe4ab\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") "
Jul 17 06:34:13.988758 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988573    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-podinfo\") pod \"f83520fc-8187-4700-93c3-47f4adbfe4ab\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") "
Jul 17 06:34:13.988758 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988613    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlnr7\" (UniqueName: \"kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-kube-api-access-jlnr7\") pod \"f83520fc-8187-4700-93c3-47f4adbfe4ab\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") "
Jul 17 06:34:13.988758 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988645    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f83520fc-8187-4700-93c3-47f4adbfe4ab-istiod-ca-cert\") pod \"f83520fc-8187-4700-93c3-47f4adbfe4ab\" (UID: \"f83520fc-8187-4700-93c3-47f4adbfe4ab\") "
Jul 17 06:34:13.989072 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988877    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f83520fc-8187-4700-93c3-47f4adbfe4ab" (UID: "f83520fc-8187-4700-93c3-47f4adbfe4ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:34:13.989072 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988882    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f83520fc-8187-4700-93c3-47f4adbfe4ab" (UID: "f83520fc-8187-4700-93c3-47f4adbfe4ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:34:13.989072 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.988902    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f83520fc-8187-4700-93c3-47f4adbfe4ab" (UID: "f83520fc-8187-4700-93c3-47f4adbfe4ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:34:13.989228 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.989138    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:13.989228 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.989160    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:13.989228 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.989166    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f83520fc-8187-4700-93c3-47f4adbfe4ab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f83520fc-8187-4700-93c3-47f4adbfe4ab" (UID: "f83520fc-8187-4700-93c3-47f4adbfe4ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:34:13.989228 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.989176    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:13.989643 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.989619    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-data" (OuterVolumeSpecName: "istio-data") pod "f83520fc-8187-4700-93c3-47f4adbfe4ab" (UID: "f83520fc-8187-4700-93c3-47f4adbfe4ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:34:13.991104 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.991075    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-token" (OuterVolumeSpecName: "istio-token") pod "f83520fc-8187-4700-93c3-47f4adbfe4ab" (UID: "f83520fc-8187-4700-93c3-47f4adbfe4ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:34:13.991208 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.991115    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-kube-api-access-jlnr7" (OuterVolumeSpecName: "kube-api-access-jlnr7") pod "f83520fc-8187-4700-93c3-47f4adbfe4ab" (UID: "f83520fc-8187-4700-93c3-47f4adbfe4ab"). InnerVolumeSpecName "kube-api-access-jlnr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:34:13.991326 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.991311    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f83520fc-8187-4700-93c3-47f4adbfe4ab" (UID: "f83520fc-8187-4700-93c3-47f4adbfe4ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:34:13.991370 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:13.991352    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f83520fc-8187-4700-93c3-47f4adbfe4ab" (UID: "f83520fc-8187-4700-93c3-47f4adbfe4ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:34:14.089908 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.089881    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:14.089908 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.089905    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlnr7\" (UniqueName: \"kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-kube-api-access-jlnr7\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:14.089908 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.089915    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f83520fc-8187-4700-93c3-47f4adbfe4ab-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:14.090156 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.089924    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:14.090156 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.089932    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:14.090156 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.089940    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f83520fc-8187-4700-93c3-47f4adbfe4ab-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:14.275181 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.275095    2558 generic.go:356] "Generic (PLEG): container finished" podID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerID="d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5" exitCode=0
Jul 17 06:34:14.275181 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.275147    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"
Jul 17 06:34:14.275181 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.275167    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" event={"ID":"f83520fc-8187-4700-93c3-47f4adbfe4ab","Type":"ContainerDied","Data":"d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5"}
Jul 17 06:34:14.275430 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.275220    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j" event={"ID":"f83520fc-8187-4700-93c3-47f4adbfe4ab","Type":"ContainerDied","Data":"c8c5379e3684398ca8a678f8e8cc135985de98be65092c169e72bd6a35e13ce0"}
Jul 17 06:34:14.275430 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.275254    2558 scope.go:117] "RemoveContainer" containerID="d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5"
Jul 17 06:34:14.284754 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.284737    2558 scope.go:117] "RemoveContainer" containerID="7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2"
Jul 17 06:34:14.292793 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.292775    2558 scope.go:117] "RemoveContainer" containerID="d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5"
Jul 17 06:34:14.293055 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:34:14.293034    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5\": container with ID starting with d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5 not found: ID does not exist" containerID="d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5"
Jul 17 06:34:14.293146 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.293068    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5"} err="failed to get container status \"d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5\": rpc error: code = NotFound desc = could not find container \"d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5\": container with ID starting with d234b2c08e0b28904332eb0418e97cf04de0332bae33e1d65bd1c1bbfebc06c5 not found: ID does not exist"
Jul 17 06:34:14.293146 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.293091    2558 scope.go:117] "RemoveContainer" containerID="7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2"
Jul 17 06:34:14.293361 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:34:14.293342    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2\": container with ID starting with 7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2 not found: ID does not exist" containerID="7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2"
Jul 17 06:34:14.293402 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.293368    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2"} err="failed to get container status \"7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2\": rpc error: code = NotFound desc = could not find container \"7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2\": container with ID starting with 7ed6fe6478a855919daac8fdd8410b92213d1f273dc7d101dd0c1a0a6fc3efb2 not found: ID does not exist"
Jul 17 06:34:14.330252 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.330233    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"]
Jul 17 06:34:14.337168 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:14.337149    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e77d3d-openshift-default-cd8bf4c69-xth7j"]
Jul 17 06:34:16.079096 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:16.079064    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" path="/var/lib/kubelet/pods/f83520fc-8187-4700-93c3-47f4adbfe4ab/volumes"
Jul 17 06:34:31.449891 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:31.449858    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"]
Jul 17 06:34:31.450316 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:31.450078    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy" containerID="cri-o://e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e" gracePeriod=30
Jul 17 06:34:32.393771 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.393738    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"]
Jul 17 06:34:32.394158 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394146    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerName="revoke-keys"
Jul 17 06:34:32.394223 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394159    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerName="revoke-keys"
Jul 17 06:34:32.394223 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394172    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerName="revoke-keys"
Jul 17 06:34:32.394223 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394178    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerName="revoke-keys"
Jul 17 06:34:32.394223 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394193    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerName="revoke-keys"
Jul 17 06:34:32.394223 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394198    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerName="revoke-keys"
Jul 17 06:34:32.394223 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394205    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerName="revoke-keys"
Jul 17 06:34:32.394223 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394210    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerName="revoke-keys"
Jul 17 06:34:32.394223 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394215    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerName="revoke-keys"
Jul 17 06:34:32.394223 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394220    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerName="revoke-keys"
Jul 17 06:34:32.394566 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394228    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy"
Jul 17 06:34:32.394566 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394233    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy"
Jul 17 06:34:32.394566 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394292    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerName="revoke-keys"
Jul 17 06:34:32.394566 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394298    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy"
Jul 17 06:34:32.394566 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394304    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerName="revoke-keys"
Jul 17 06:34:32.394566 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394310    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerName="revoke-keys"
Jul 17 06:34:32.394566 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394317    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerName="revoke-keys"
Jul 17 06:34:32.394566 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.394323    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy"
Jul 17 06:34:32.396434 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.396420    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.399917 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.399891    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f90cce-openshift-default-dockercfg-9k2r2\""
Jul 17 06:34:32.413296 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.413276    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"]
Jul 17 06:34:32.457581 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.457544    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-certs\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.458058 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.457672    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-istio-token\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.458058 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.457767    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-data\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.458058 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.457825    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2gwb\" (UniqueName: \"kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-kube-api-access-b2gwb\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.458058 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.457854    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8fab551-ae71-4a24-b398-f95931b88175-istio-podinfo\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.458058 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.457949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8fab551-ae71-4a24-b398-f95931b88175-istiod-ca-cert\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.458058 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.457999    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-envoy\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.458058 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.458033    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-socket\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.458453 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.458076    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-credential-socket\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.559551 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.559519    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-istio-token\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.559726 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.559573    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-data\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.559726 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.559607    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2gwb\" (UniqueName: \"kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-kube-api-access-b2gwb\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.559726 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.559623    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8fab551-ae71-4a24-b398-f95931b88175-istio-podinfo\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.559726 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.559659    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8fab551-ae71-4a24-b398-f95931b88175-istiod-ca-cert\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.559726 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.559690    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-envoy\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.559726 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.559718    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-socket\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.560008 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.559754    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-credential-socket\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.560008 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.559793    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-certs\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.560112 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.560049    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-data\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.560202 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.560177    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-socket\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.560278 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.560254    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-credential-socket\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.560344 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.560301    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-certs\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.560389 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.560365    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8fab551-ae71-4a24-b398-f95931b88175-istiod-ca-cert\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.562205 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.562181    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-envoy\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.562289 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.562272    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8fab551-ae71-4a24-b398-f95931b88175-istio-podinfo\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.577717 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.577691    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2gwb\" (UniqueName: \"kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-kube-api-access-b2gwb\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.577942 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.577927    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-istio-token\") pod \"e2e-api-a-f90cce-openshift-default-699c76499f-p58pb\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") " pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.707078 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.706982    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:32.872206 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.872166    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"]
Jul 17 06:34:32.881895 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:34:32.881870    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8fab551_ae71_4a24_b398_f95931b88175.slice/crio-ed8c7d0415262b14e3687c721fd99bd9818cd9e139bcfb3bc798606917dcb5c8 WatchSource:0}: Error finding container ed8c7d0415262b14e3687c721fd99bd9818cd9e139bcfb3bc798606917dcb5c8: Status 404 returned error can't find the container with id ed8c7d0415262b14e3687c721fd99bd9818cd9e139bcfb3bc798606917dcb5c8
Jul 17 06:34:32.883990 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.883954    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:32.884065 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.884028    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:32.884065 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:32.884057    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:33.378661 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:33.378628    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" event={"ID":"f8fab551-ae71-4a24-b398-f95931b88175","Type":"ContainerStarted","Data":"2d6c076da0468ccbc2a5be0764d9a294e25df59cf1b59a7355ffa0f0162cad76"}
Jul 17 06:34:33.378833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:33.378679    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" event={"ID":"f8fab551-ae71-4a24-b398-f95931b88175","Type":"ContainerStarted","Data":"ed8c7d0415262b14e3687c721fd99bd9818cd9e139bcfb3bc798606917dcb5c8"}
Jul 17 06:34:33.411571 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:33.411523    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" podStartSLOduration=1.411508046 podStartE2EDuration="1.411508046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:34:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:34:33.409486088 +0000 UTC m=+3195.851424654" watchObservedRunningTime="2026-07-17 06:34:33.411508046 +0000 UTC m=+3195.853446620"
Jul 17 06:34:33.708020 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:33.707928    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:33.712995 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:33.712972    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:34.384627 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:34.384592    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:34.385640 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:34.385618    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:36.718037 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.717958    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:34:36.801590 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.801549    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-socket\") pod \"3022e42b-791d-4281-a4f1-2bb70558bfeb\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") "
Jul 17 06:34:36.801796 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.801603    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-certs\") pod \"3022e42b-791d-4281-a4f1-2bb70558bfeb\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") "
Jul 17 06:34:36.801796 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.801622    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3022e42b-791d-4281-a4f1-2bb70558bfeb-istiod-ca-cert\") pod \"3022e42b-791d-4281-a4f1-2bb70558bfeb\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") "
Jul 17 06:34:36.801796 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.801667    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-envoy\") pod \"3022e42b-791d-4281-a4f1-2bb70558bfeb\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") "
Jul 17 06:34:36.801796 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.801708    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rr52z\" (UniqueName: \"kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-kube-api-access-rr52z\") pod \"3022e42b-791d-4281-a4f1-2bb70558bfeb\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") "
Jul 17 06:34:36.801796 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.801731    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-credential-socket\") pod \"3022e42b-791d-4281-a4f1-2bb70558bfeb\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") "
Jul 17 06:34:36.801796 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.801779    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-podinfo\") pod \"3022e42b-791d-4281-a4f1-2bb70558bfeb\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") "
Jul 17 06:34:36.802128 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.801810    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-token\") pod \"3022e42b-791d-4281-a4f1-2bb70558bfeb\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") "
Jul 17 06:34:36.802128 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.801857    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-data\") pod \"3022e42b-791d-4281-a4f1-2bb70558bfeb\" (UID: \"3022e42b-791d-4281-a4f1-2bb70558bfeb\") "
Jul 17 06:34:36.802128 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.802101    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3022e42b-791d-4281-a4f1-2bb70558bfeb" (UID: "3022e42b-791d-4281-a4f1-2bb70558bfeb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:34:36.802296 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.802222    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3022e42b-791d-4281-a4f1-2bb70558bfeb" (UID: "3022e42b-791d-4281-a4f1-2bb70558bfeb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:34:36.802296 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.802263    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:36.802562 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.802530    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3022e42b-791d-4281-a4f1-2bb70558bfeb" (UID: "3022e42b-791d-4281-a4f1-2bb70558bfeb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:34:36.802859 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.802638    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3022e42b-791d-4281-a4f1-2bb70558bfeb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3022e42b-791d-4281-a4f1-2bb70558bfeb" (UID: "3022e42b-791d-4281-a4f1-2bb70558bfeb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:34:36.803282 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.803254    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-data" (OuterVolumeSpecName: "istio-data") pod "3022e42b-791d-4281-a4f1-2bb70558bfeb" (UID: "3022e42b-791d-4281-a4f1-2bb70558bfeb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:34:36.804331 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.804312    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3022e42b-791d-4281-a4f1-2bb70558bfeb" (UID: "3022e42b-791d-4281-a4f1-2bb70558bfeb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:34:36.804461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.804447    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-token" (OuterVolumeSpecName: "istio-token") pod "3022e42b-791d-4281-a4f1-2bb70558bfeb" (UID: "3022e42b-791d-4281-a4f1-2bb70558bfeb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:34:36.804676 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.804652    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-kube-api-access-rr52z" (OuterVolumeSpecName: "kube-api-access-rr52z") pod "3022e42b-791d-4281-a4f1-2bb70558bfeb" (UID: "3022e42b-791d-4281-a4f1-2bb70558bfeb"). InnerVolumeSpecName "kube-api-access-rr52z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:34:36.804676 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.804657    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3022e42b-791d-4281-a4f1-2bb70558bfeb" (UID: "3022e42b-791d-4281-a4f1-2bb70558bfeb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:34:36.903041 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.902997    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:36.903041 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.903036    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3022e42b-791d-4281-a4f1-2bb70558bfeb-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:36.903041 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.903045    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:36.903276 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.903054    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr52z\" (UniqueName: \"kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-kube-api-access-rr52z\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:36.903276 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.903063    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:36.903276 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.903071    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:36.903276 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.903080    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:36.903276 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:36.903087    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3022e42b-791d-4281-a4f1-2bb70558bfeb-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:34:37.403020 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.402979    2558 generic.go:356] "Generic (PLEG): container finished" podID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerID="e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e" exitCode=0
Jul 17 06:34:37.403206 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.403042    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"
Jul 17 06:34:37.403206 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.403061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" event={"ID":"3022e42b-791d-4281-a4f1-2bb70558bfeb","Type":"ContainerDied","Data":"e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e"}
Jul 17 06:34:37.403206 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.403128    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd" event={"ID":"3022e42b-791d-4281-a4f1-2bb70558bfeb","Type":"ContainerDied","Data":"9878588321e6bb2284e63b107a785e4c5e7f5ec975380e77f07df0edcee2d26f"}
Jul 17 06:34:37.403206 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.403159    2558 scope.go:117] "RemoveContainer" containerID="e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e"
Jul 17 06:34:37.413791 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.413774    2558 scope.go:117] "RemoveContainer" containerID="37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e"
Jul 17 06:34:37.423380 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.423355    2558 scope.go:117] "RemoveContainer" containerID="e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e"
Jul 17 06:34:37.423712 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:34:37.423691    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e\": container with ID starting with e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e not found: ID does not exist" containerID="e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e"
Jul 17 06:34:37.423788 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.423722    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e"} err="failed to get container status \"e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e\": rpc error: code = NotFound desc = could not find container \"e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e\": container with ID starting with e4293a018e4ac80425f5717b92747185f9f044ead273b522598a3e5b38b7420e not found: ID does not exist"
Jul 17 06:34:37.423788 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.423740    2558 scope.go:117] "RemoveContainer" containerID="37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e"
Jul 17 06:34:37.424021 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:34:37.424005    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e\": container with ID starting with 37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e not found: ID does not exist" containerID="37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e"
Jul 17 06:34:37.424067 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.424029    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e"} err="failed to get container status \"37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e\": rpc error: code = NotFound desc = could not find container \"37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e\": container with ID starting with 37ea00b5ddcac41979b15f478bdc873c077f98b51c7a173f504d910964c8925e not found: ID does not exist"
Jul 17 06:34:37.433282 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.433256    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"]
Jul 17 06:34:37.441515 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:37.441488    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-142438-openshift-default-6db89c8b99-dbkgd"]
Jul 17 06:34:38.078742 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:38.078709    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" path="/var/lib/kubelet/pods/3022e42b-791d-4281-a4f1-2bb70558bfeb/volumes"
Jul 17 06:34:40.778898 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.778863    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"]
Jul 17 06:34:40.779375 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779326    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy"
Jul 17 06:34:40.779375 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779339    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy"
Jul 17 06:34:40.779375 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779355    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerName="revoke-keys"
Jul 17 06:34:40.779375 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779360    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerName="revoke-keys"
Jul 17 06:34:40.779375 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779367    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy"
Jul 17 06:34:40.779375 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779372    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy"
Jul 17 06:34:40.779601 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779384    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy"
Jul 17 06:34:40.779601 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779389    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83520fc-8187-4700-93c3-47f4adbfe4ab" containerName="istio-proxy"
Jul 17 06:34:40.779601 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779452    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy"
Jul 17 06:34:40.779601 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779464    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3022e42b-791d-4281-a4f1-2bb70558bfeb" containerName="istio-proxy"
Jul 17 06:34:40.779601 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779493    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="427b49fe-c184-4cbc-b40d-2a0c4754c387" containerName="revoke-keys"
Jul 17 06:34:40.779601 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.779499    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23032bf-d45a-4fdf-80fa-17f2e07159f0" containerName="revoke-keys"
Jul 17 06:34:40.782596 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.782574    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.788921 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.788894    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1b3bd1-openshift-default-dockercfg-xdw5t\""
Jul 17 06:34:40.813163 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.813122    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"]
Jul 17 06:34:40.862761 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.862560    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-token\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.862983 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.862774    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istiod-ca-cert\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.862983 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.862838    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-data\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.862983 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.862874    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-certs\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.862983 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.862909    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-credential-socket\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.862983 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.862941    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-podinfo\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.863277 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.863024    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrqbb\" (UniqueName: \"kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-kube-api-access-xrqbb\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.863277 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.863105    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-envoy\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.863277 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.863177    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-socket\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964076 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964031    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-envoy\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964308 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964111    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-socket\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964308 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964162    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-token\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964308 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964184    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istiod-ca-cert\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964308 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964224    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-data\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964308 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964255    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-certs\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964308 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964284    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-credential-socket\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964310    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-podinfo\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964626    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrqbb\" (UniqueName: \"kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-kube-api-access-xrqbb\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964667    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-socket\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964873 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964760    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-credential-socket\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.964873 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.964800    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-data\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.965077 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.965056    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-certs\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.965233 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.965177    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istiod-ca-cert\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.966735 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.966712    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-envoy\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.966956 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.966938    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-podinfo\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.979590 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.979552    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrqbb\" (UniqueName: \"kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-kube-api-access-xrqbb\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:40.980524 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:40.980504    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-token\") pod \"e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") " pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:41.093666 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:41.093611    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:41.312123 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:41.312095    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"]
Jul 17 06:34:41.315112 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:34:41.315079    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0bce2f_ccc1_4b87_93a3_4bb87565a303.slice/crio-398ba14e4bcf2b9ec92838c3973118bf41eecb06e1ebd794f17ecf8f0d031707 WatchSource:0}: Error finding container 398ba14e4bcf2b9ec92838c3973118bf41eecb06e1ebd794f17ecf8f0d031707: Status 404 returned error can't find the container with id 398ba14e4bcf2b9ec92838c3973118bf41eecb06e1ebd794f17ecf8f0d031707
Jul 17 06:34:41.317228 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:41.317196    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:41.317303 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:41.317258    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:41.317303 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:41.317286    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:41.442500 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:41.440968    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" event={"ID":"1e0bce2f-ccc1-4b87-93a3-4bb87565a303","Type":"ContainerStarted","Data":"e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39"}
Jul 17 06:34:41.442500 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:41.441043    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" event={"ID":"1e0bce2f-ccc1-4b87-93a3-4bb87565a303","Type":"ContainerStarted","Data":"398ba14e4bcf2b9ec92838c3973118bf41eecb06e1ebd794f17ecf8f0d031707"}
Jul 17 06:34:41.473552 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:41.473483    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" podStartSLOduration=1.473442365 podStartE2EDuration="1.473442365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:34:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:34:41.470415533 +0000 UTC m=+3203.912354128" watchObservedRunningTime="2026-07-17 06:34:41.473442365 +0000 UTC m=+3203.915380941"
Jul 17 06:34:42.094105 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:42.094066    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:42.099443 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:42.099414    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:42.450137 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:42.449905    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:42.453987 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:42.453957    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:47.478994 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:47.478960    2558 generic.go:356] "Generic (PLEG): container finished" podID="f8fab551-ae71-4a24-b398-f95931b88175" containerID="2d6c076da0468ccbc2a5be0764d9a294e25df59cf1b59a7355ffa0f0162cad76" exitCode=0
Jul 17 06:34:47.479433 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:47.479040    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" event={"ID":"f8fab551-ae71-4a24-b398-f95931b88175","Type":"ContainerDied","Data":"2d6c076da0468ccbc2a5be0764d9a294e25df59cf1b59a7355ffa0f0162cad76"}
Jul 17 06:34:47.479500 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:47.479457    2558 scope.go:117] "RemoveContainer" containerID="2d6c076da0468ccbc2a5be0764d9a294e25df59cf1b59a7355ffa0f0162cad76"
Jul 17 06:34:47.480556 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:47.480531    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:47.480641 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:47.480596    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:47.480641 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:47.480627    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:48.486781 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:48.486742    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" event={"ID":"f8fab551-ae71-4a24-b398-f95931b88175","Type":"ContainerStarted","Data":"6f03f05c17742c114f3d5ea3e05f1dfe679fcb627d9d4bf52922b8bee5c7c958"}
Jul 17 06:34:48.708079 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:48.708043    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:48.709102 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:48.709078    2558 patch_prober.go:28] interesting pod/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Jul 17 06:34:48.709198 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:48.709160    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" podUID="f8fab551-ae71-4a24-b398-f95931b88175" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Jul 17 06:34:49.384919 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:49.384862    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:49.708398 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:49.708304    2558 patch_prober.go:28] interesting pod/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Jul 17 06:34:49.708398 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:49.708361    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" podUID="f8fab551-ae71-4a24-b398-f95931b88175" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Jul 17 06:34:50.708198 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:50.708161    2558 patch_prober.go:28] interesting pod/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Jul 17 06:34:50.708391 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:50.708217    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" podUID="f8fab551-ae71-4a24-b398-f95931b88175" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Jul 17 06:34:51.707725 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:51.707688    2558 patch_prober.go:28] interesting pod/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Jul 17 06:34:51.708109 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:51.707747    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" podUID="f8fab551-ae71-4a24-b398-f95931b88175" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Jul 17 06:34:52.711643 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:52.711608    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:52.712540 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:52.712516    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:34:57.451424 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:57.451378    2558 patch_prober.go:28] interesting pod/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Jul 17 06:34:57.451879 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:57.451444    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Jul 17 06:34:57.537635 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:57.537600    2558 generic.go:356] "Generic (PLEG): container finished" podID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerID="e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39" exitCode=0
Jul 17 06:34:57.537833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:57.537682    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" event={"ID":"1e0bce2f-ccc1-4b87-93a3-4bb87565a303","Type":"ContainerDied","Data":"e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39"}
Jul 17 06:34:57.538151 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:57.538136    2558 scope.go:117] "RemoveContainer" containerID="e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39"
Jul 17 06:34:57.539194 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:57.539163    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:57.539313 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:57.539250    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:57.539313 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:57.539302    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:34:58.550645 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:58.550600    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" event={"ID":"1e0bce2f-ccc1-4b87-93a3-4bb87565a303","Type":"ContainerStarted","Data":"43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0"}
Jul 17 06:34:59.094032 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:59.093996    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:34:59.094250 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:59.094228    2558 patch_prober.go:28] interesting pod/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Jul 17 06:34:59.094300 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:34:59.094279    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Jul 17 06:35:00.094009 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:00.093970    2558 patch_prober.go:28] interesting pod/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Jul 17 06:35:00.094427 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:00.094033    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Jul 17 06:35:01.094350 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:01.094310    2558 patch_prober.go:28] interesting pod/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Jul 17 06:35:01.094798 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:01.094370    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Jul 17 06:35:02.094673 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:02.094638    2558 patch_prober.go:28] interesting pod/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Jul 17 06:35:02.095099 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:02.094696    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Jul 17 06:35:03.098289 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:03.098253    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:35:03.098792 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:03.098699    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:35:03.099420 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:03.099402    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:35:23.589052 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:23.589018    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"]
Jul 17 06:35:23.589516 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:23.589220    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" podUID="f8fab551-ae71-4a24-b398-f95931b88175" containerName="istio-proxy" containerID="cri-o://6f03f05c17742c114f3d5ea3e05f1dfe679fcb627d9d4bf52922b8bee5c7c958" gracePeriod=30
Jul 17 06:35:28.720853 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.720816    2558 generic.go:356] "Generic (PLEG): container finished" podID="f8fab551-ae71-4a24-b398-f95931b88175" containerID="6f03f05c17742c114f3d5ea3e05f1dfe679fcb627d9d4bf52922b8bee5c7c958" exitCode=0
Jul 17 06:35:28.721289 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.720853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" event={"ID":"f8fab551-ae71-4a24-b398-f95931b88175","Type":"ContainerDied","Data":"6f03f05c17742c114f3d5ea3e05f1dfe679fcb627d9d4bf52922b8bee5c7c958"}
Jul 17 06:35:28.721289 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.720951    2558 scope.go:117] "RemoveContainer" containerID="2d6c076da0468ccbc2a5be0764d9a294e25df59cf1b59a7355ffa0f0162cad76"
Jul 17 06:35:28.850133 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.850077    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:35:28.925854 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.925822    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8fab551-ae71-4a24-b398-f95931b88175-istio-podinfo\") pod \"f8fab551-ae71-4a24-b398-f95931b88175\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") "
Jul 17 06:35:28.925854 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.925856    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-istio-token\") pod \"f8fab551-ae71-4a24-b398-f95931b88175\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") "
Jul 17 06:35:28.926096 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.925877    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-certs\") pod \"f8fab551-ae71-4a24-b398-f95931b88175\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") "
Jul 17 06:35:28.926096 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.925900    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8fab551-ae71-4a24-b398-f95931b88175-istiod-ca-cert\") pod \"f8fab551-ae71-4a24-b398-f95931b88175\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") "
Jul 17 06:35:28.926096 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.925955    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-envoy\") pod \"f8fab551-ae71-4a24-b398-f95931b88175\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") "
Jul 17 06:35:28.926096 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.926020    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-data\") pod \"f8fab551-ae71-4a24-b398-f95931b88175\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") "
Jul 17 06:35:28.926359 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.926072    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2gwb\" (UniqueName: \"kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-kube-api-access-b2gwb\") pod \"f8fab551-ae71-4a24-b398-f95931b88175\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") "
Jul 17 06:35:28.926359 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.926162    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f8fab551-ae71-4a24-b398-f95931b88175" (UID: "f8fab551-ae71-4a24-b398-f95931b88175"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:35:28.926359 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.926223    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-socket\") pod \"f8fab551-ae71-4a24-b398-f95931b88175\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") "
Jul 17 06:35:28.926359 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.926258    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-credential-socket\") pod \"f8fab551-ae71-4a24-b398-f95931b88175\" (UID: \"f8fab551-ae71-4a24-b398-f95931b88175\") "
Jul 17 06:35:28.926603 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.926394    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8fab551-ae71-4a24-b398-f95931b88175-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f8fab551-ae71-4a24-b398-f95931b88175" (UID: "f8fab551-ae71-4a24-b398-f95931b88175"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:35:28.926603 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.926528    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f8fab551-ae71-4a24-b398-f95931b88175" (UID: "f8fab551-ae71-4a24-b398-f95931b88175"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:35:28.926875 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.926793    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:28.926875 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.926816    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:28.926875 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.926831    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8fab551-ae71-4a24-b398-f95931b88175-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:28.927239 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.927192    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f8fab551-ae71-4a24-b398-f95931b88175" (UID: "f8fab551-ae71-4a24-b398-f95931b88175"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:35:28.927462 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.927427    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-data" (OuterVolumeSpecName: "istio-data") pod "f8fab551-ae71-4a24-b398-f95931b88175" (UID: "f8fab551-ae71-4a24-b398-f95931b88175"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:35:28.929090 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.929048    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f8fab551-ae71-4a24-b398-f95931b88175-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f8fab551-ae71-4a24-b398-f95931b88175" (UID: "f8fab551-ae71-4a24-b398-f95931b88175"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:35:28.929090 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.929082    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f8fab551-ae71-4a24-b398-f95931b88175" (UID: "f8fab551-ae71-4a24-b398-f95931b88175"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:35:28.929250 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.929222    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-istio-token" (OuterVolumeSpecName: "istio-token") pod "f8fab551-ae71-4a24-b398-f95931b88175" (UID: "f8fab551-ae71-4a24-b398-f95931b88175"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:35:28.929347 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:28.929329    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-kube-api-access-b2gwb" (OuterVolumeSpecName: "kube-api-access-b2gwb") pod "f8fab551-ae71-4a24-b398-f95931b88175" (UID: "f8fab551-ae71-4a24-b398-f95931b88175"). InnerVolumeSpecName "kube-api-access-b2gwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:35:29.027896 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:29.027867    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:29.027896 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:29.027895    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2gwb\" (UniqueName: \"kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-kube-api-access-b2gwb\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:29.028049 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:29.027912    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:29.028049 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:29.027924    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8fab551-ae71-4a24-b398-f95931b88175-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:29.028049 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:29.027936    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8fab551-ae71-4a24-b398-f95931b88175-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:29.028049 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:29.027964    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8fab551-ae71-4a24-b398-f95931b88175-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:29.728328 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:29.728296    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"
Jul 17 06:35:29.728864 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:29.728293    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb" event={"ID":"f8fab551-ae71-4a24-b398-f95931b88175","Type":"ContainerDied","Data":"ed8c7d0415262b14e3687c721fd99bd9818cd9e139bcfb3bc798606917dcb5c8"}
Jul 17 06:35:29.728864 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:29.728437    2558 scope.go:117] "RemoveContainer" containerID="6f03f05c17742c114f3d5ea3e05f1dfe679fcb627d9d4bf52922b8bee5c7c958"
Jul 17 06:35:29.751964 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:29.751939    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"]
Jul 17 06:35:29.756233 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:29.756212    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f90cce-openshift-default-699c76499f-p58pb"]
Jul 17 06:35:30.078499 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:30.078408    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8fab551-ae71-4a24-b398-f95931b88175" path="/var/lib/kubelet/pods/f8fab551-ae71-4a24-b398-f95931b88175/volumes"
Jul 17 06:35:50.593188 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:50.593140    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"]
Jul 17 06:35:50.593746 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:50.593376    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy" containerID="cri-o://43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0" gracePeriod=30
Jul 17 06:35:51.503571 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.503542    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"]
Jul 17 06:35:51.504011 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.503999    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8fab551-ae71-4a24-b398-f95931b88175" containerName="istio-proxy"
Jul 17 06:35:51.504060 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.504014    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8fab551-ae71-4a24-b398-f95931b88175" containerName="istio-proxy"
Jul 17 06:35:51.504060 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.504024    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8fab551-ae71-4a24-b398-f95931b88175" containerName="istio-proxy"
Jul 17 06:35:51.504060 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.504034    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8fab551-ae71-4a24-b398-f95931b88175" containerName="istio-proxy"
Jul 17 06:35:51.504152 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.504098    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8fab551-ae71-4a24-b398-f95931b88175" containerName="istio-proxy"
Jul 17 06:35:51.507505 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.507462    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.510040 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.510021    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-ff4257-openshift-default-dockercfg-br46c\""
Jul 17 06:35:51.521927 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.521894    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"]
Jul 17 06:35:51.529994 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.529956    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-istio-token\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.530099 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.530007    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-credential-socket\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.530099 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.530064    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-data\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.530224 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.530205    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-socket\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.530280 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.530264    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-certs\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.530314 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.530298    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/785df468-28af-4515-b0f0-5c97f5f588dc-istio-podinfo\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.530348 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.530329    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92cws\" (UniqueName: \"kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-kube-api-access-92cws\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.530385 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.530366    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-envoy\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.530430 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.530417    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/785df468-28af-4515-b0f0-5c97f5f588dc-istiod-ca-cert\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631001 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.630969    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-istio-token\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631001 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631004    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-credential-socket\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631564 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631025    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-data\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631564 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631089    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-socket\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631564 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631119    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-certs\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631564 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631144    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/785df468-28af-4515-b0f0-5c97f5f588dc-istio-podinfo\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631564 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631177    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92cws\" (UniqueName: \"kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-kube-api-access-92cws\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631564 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631212    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-envoy\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631564 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631256    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/785df468-28af-4515-b0f0-5c97f5f588dc-istiod-ca-cert\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631564 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631453    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-credential-socket\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631564 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631515    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-data\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631986 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631767    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-socket\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631986 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631860    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-certs\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.631986 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.631937    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/785df468-28af-4515-b0f0-5c97f5f588dc-istiod-ca-cert\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.634064 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.634038    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-envoy\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.634196 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.634066    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/785df468-28af-4515-b0f0-5c97f5f588dc-istio-podinfo\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.644858 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.644837    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92cws\" (UniqueName: \"kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-kube-api-access-92cws\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.647084 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.647062    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-istio-token\") pod \"e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") " pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.817819 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.817748    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:51.967305 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.967280    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"]
Jul 17 06:35:51.969441 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:35:51.969410    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785df468_28af_4515_b0f0_5c97f5f588dc.slice/crio-22f5dabc87e98e9fc75a1f77e3f4d0450b9a2a2358aa52338a5c224327ea1238 WatchSource:0}: Error finding container 22f5dabc87e98e9fc75a1f77e3f4d0450b9a2a2358aa52338a5c224327ea1238: Status 404 returned error can't find the container with id 22f5dabc87e98e9fc75a1f77e3f4d0450b9a2a2358aa52338a5c224327ea1238
Jul 17 06:35:51.971607 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.971574    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:35:51.971796 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.971654    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:35:51.971796 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:51.971702    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:35:52.860962 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:52.860924    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" event={"ID":"785df468-28af-4515-b0f0-5c97f5f588dc","Type":"ContainerStarted","Data":"de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239"}
Jul 17 06:35:52.861559 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:52.860992    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" event={"ID":"785df468-28af-4515-b0f0-5c97f5f588dc","Type":"ContainerStarted","Data":"22f5dabc87e98e9fc75a1f77e3f4d0450b9a2a2358aa52338a5c224327ea1238"}
Jul 17 06:35:52.881553 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:52.881495    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" podStartSLOduration=1.881450984 podStartE2EDuration="1.881450984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:35:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:35:52.881198394 +0000 UTC m=+3275.323136973" watchObservedRunningTime="2026-07-17 06:35:52.881450984 +0000 UTC m=+3275.323389560"
Jul 17 06:35:53.818238 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:53.818194    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:53.823377 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:53.823348    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:53.874245 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:53.874200    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:53.875451 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:53.875428    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:35:55.885301 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.885268    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:35:55.887672 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.887642    2558 generic.go:356] "Generic (PLEG): container finished" podID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerID="43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0" exitCode=0
Jul 17 06:35:55.887833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.887696    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"
Jul 17 06:35:55.887833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.887724    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" event={"ID":"1e0bce2f-ccc1-4b87-93a3-4bb87565a303","Type":"ContainerDied","Data":"43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0"}
Jul 17 06:35:55.887833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.887795    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v" event={"ID":"1e0bce2f-ccc1-4b87-93a3-4bb87565a303","Type":"ContainerDied","Data":"398ba14e4bcf2b9ec92838c3973118bf41eecb06e1ebd794f17ecf8f0d031707"}
Jul 17 06:35:55.887833 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.887829    2558 scope.go:117] "RemoveContainer" containerID="43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0"
Jul 17 06:35:55.897249 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.897225    2558 scope.go:117] "RemoveContainer" containerID="e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39"
Jul 17 06:35:55.905991 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.905963    2558 scope.go:117] "RemoveContainer" containerID="43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0"
Jul 17 06:35:55.906272 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:35:55.906252    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0\": container with ID starting with 43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0 not found: ID does not exist" containerID="43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0"
Jul 17 06:35:55.906344 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.906281    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0"} err="failed to get container status \"43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0\": rpc error: code = NotFound desc = could not find container \"43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0\": container with ID starting with 43edcddb52a13cca988705585e3921350685b2d58d6dfd2e2ba39058621046a0 not found: ID does not exist"
Jul 17 06:35:55.906344 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.906299    2558 scope.go:117] "RemoveContainer" containerID="e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39"
Jul 17 06:35:55.906584 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:35:55.906562    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39\": container with ID starting with e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39 not found: ID does not exist" containerID="e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39"
Jul 17 06:35:55.906646 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.906590    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39"} err="failed to get container status \"e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39\": rpc error: code = NotFound desc = could not find container \"e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39\": container with ID starting with e908962d911c2c485516581358827e3ea78f69244ccc05003eac673a4a878e39 not found: ID does not exist"
Jul 17 06:35:55.976998 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.976964    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-podinfo\") pod \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") "
Jul 17 06:35:55.976998 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977000    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-data\") pod \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") "
Jul 17 06:35:55.977239 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977060    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrqbb\" (UniqueName: \"kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-kube-api-access-xrqbb\") pod \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") "
Jul 17 06:35:55.977239 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977094    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-token\") pod \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") "
Jul 17 06:35:55.977239 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977124    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-credential-socket\") pod \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") "
Jul 17 06:35:55.977239 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977207    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-envoy\") pod \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") "
Jul 17 06:35:55.977425 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977245    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istiod-ca-cert\") pod \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") "
Jul 17 06:35:55.977425 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977278    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-socket\") pod \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") "
Jul 17 06:35:55.977425 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977314    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-certs\") pod \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\" (UID: \"1e0bce2f-ccc1-4b87-93a3-4bb87565a303\") "
Jul 17 06:35:55.977628 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977462    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1e0bce2f-ccc1-4b87-93a3-4bb87565a303" (UID: "1e0bce2f-ccc1-4b87-93a3-4bb87565a303"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:35:55.977696 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977664    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1e0bce2f-ccc1-4b87-93a3-4bb87565a303" (UID: "1e0bce2f-ccc1-4b87-93a3-4bb87565a303"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:35:55.978123 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977845    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:55.978123 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.977868    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:55.978123 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.978082    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1e0bce2f-ccc1-4b87-93a3-4bb87565a303" (UID: "1e0bce2f-ccc1-4b87-93a3-4bb87565a303"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:35:55.978123 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.978107    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1e0bce2f-ccc1-4b87-93a3-4bb87565a303" (UID: "1e0bce2f-ccc1-4b87-93a3-4bb87565a303"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:35:55.978436 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.978333    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-data" (OuterVolumeSpecName: "istio-data") pod "1e0bce2f-ccc1-4b87-93a3-4bb87565a303" (UID: "1e0bce2f-ccc1-4b87-93a3-4bb87565a303"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:35:55.979652 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.979632    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1e0bce2f-ccc1-4b87-93a3-4bb87565a303" (UID: "1e0bce2f-ccc1-4b87-93a3-4bb87565a303"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:35:55.979746 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.979660    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-token" (OuterVolumeSpecName: "istio-token") pod "1e0bce2f-ccc1-4b87-93a3-4bb87565a303" (UID: "1e0bce2f-ccc1-4b87-93a3-4bb87565a303"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:35:55.980269 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.980236    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1e0bce2f-ccc1-4b87-93a3-4bb87565a303" (UID: "1e0bce2f-ccc1-4b87-93a3-4bb87565a303"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:35:55.980269 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:55.980255    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-kube-api-access-xrqbb" (OuterVolumeSpecName: "kube-api-access-xrqbb") pod "1e0bce2f-ccc1-4b87-93a3-4bb87565a303" (UID: "1e0bce2f-ccc1-4b87-93a3-4bb87565a303"). InnerVolumeSpecName "kube-api-access-xrqbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:35:56.079302 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:56.079259    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:56.079302 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:56.079293    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:56.079302 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:56.079307    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:56.079626 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:56.079319    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:56.079626 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:56.079331    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrqbb\" (UniqueName: \"kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-kube-api-access-xrqbb\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:56.079626 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:56.079343    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:56.079626 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:56.079354    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e0bce2f-ccc1-4b87-93a3-4bb87565a303-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:35:56.205222 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:56.205129    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"]
Jul 17 06:35:56.212222 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:56.212188    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1b3bd1-openshift-default-54cbf46df9-qfx6v"]
Jul 17 06:35:58.079149 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:58.079112    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" path="/var/lib/kubelet/pods/1e0bce2f-ccc1-4b87-93a3-4bb87565a303/volumes"
Jul 17 06:35:59.882603 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:59.882571    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"]
Jul 17 06:35:59.883022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:59.883008    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy"
Jul 17 06:35:59.883022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:59.883023    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy"
Jul 17 06:35:59.883116 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:59.883033    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy"
Jul 17 06:35:59.883116 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:59.883038    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy"
Jul 17 06:35:59.883184 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:59.883130    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy"
Jul 17 06:35:59.883184 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:59.883141    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8fab551-ae71-4a24-b398-f95931b88175" containerName="istio-proxy"
Jul 17 06:35:59.885587 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:59.885563    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:35:59.889155 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:59.889133    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-346c23-openshift-default-dockercfg-2wt5d\""
Jul 17 06:35:59.899691 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:35:59.899659    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"]
Jul 17 06:36:00.018228 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.018191    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-envoy\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.018405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.018234    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmxgr\" (UniqueName: \"kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-kube-api-access-xmxgr\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.018405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.018272    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-socket\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.018405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.018301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-credential-socket\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.018405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.018345    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-token\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.018405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.018389    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-podinfo\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.018651 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.018431    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414677a6-4e6e-435b-8afc-c5696f6d8393-istiod-ca-cert\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.018651 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.018453    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-data\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.018651 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.018502    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-certs\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.119715 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.119685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-certs\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.119864 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.119739    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-envoy\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.119864 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.119782    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmxgr\" (UniqueName: \"kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-kube-api-access-xmxgr\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.119864 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.119820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-socket\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.119864 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.119848    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-credential-socket\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.120065 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.119905    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-token\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.120065 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.119945    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-podinfo\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.120065 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.120004    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414677a6-4e6e-435b-8afc-c5696f6d8393-istiod-ca-cert\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.120065 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.120036    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-data\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.120283 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.120136    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-certs\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.120283 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.120202    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-socket\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.120354 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.120285    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-credential-socket\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.120389 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.120345    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-data\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.120723 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.120704    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414677a6-4e6e-435b-8afc-c5696f6d8393-istiod-ca-cert\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.122137 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.122114    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-envoy\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.122519 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.122501    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-podinfo\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.128990 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.128961    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmxgr\" (UniqueName: \"kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-kube-api-access-xmxgr\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.129085 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.129032    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-token\") pod \"e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") " pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.195497 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.195434    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:00.347443 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.347407    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"]
Jul 17 06:36:00.349208 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:36:00.349176    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414677a6_4e6e_435b_8afc_c5696f6d8393.slice/crio-ac2e088b9be2c354954a7f73234fdb187dbb489a8687e823424a259c889f4b4a WatchSource:0}: Error finding container ac2e088b9be2c354954a7f73234fdb187dbb489a8687e823424a259c889f4b4a: Status 404 returned error can't find the container with id ac2e088b9be2c354954a7f73234fdb187dbb489a8687e823424a259c889f4b4a
Jul 17 06:36:00.351579 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.351540    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:36:00.351695 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.351617    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:36:00.351695 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.351649    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:36:00.916630 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.916591    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" event={"ID":"414677a6-4e6e-435b-8afc-c5696f6d8393","Type":"ContainerStarted","Data":"d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e"}
Jul 17 06:36:00.917022 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.916650    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" event={"ID":"414677a6-4e6e-435b-8afc-c5696f6d8393","Type":"ContainerStarted","Data":"ac2e088b9be2c354954a7f73234fdb187dbb489a8687e823424a259c889f4b4a"}
Jul 17 06:36:00.940875 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:00.940801    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" podStartSLOduration=1.940780572 podStartE2EDuration="1.940780572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:35:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:36:00.938728674 +0000 UTC m=+3283.380667250" watchObservedRunningTime="2026-07-17 06:36:00.940780572 +0000 UTC m=+3283.382719146"
Jul 17 06:36:01.195742 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:01.195633    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:01.201174 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:01.201144    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:01.922626 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:01.922595    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:01.923924 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:01.923896    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:09.915656 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:09.915610    2558 patch_prober.go:28] interesting pod/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 06:36:09.916045 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:09.915688    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 06:36:10.992960 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:10.992863    2558 generic.go:356] "Generic (PLEG): container finished" podID="785df468-28af-4515-b0f0-5c97f5f588dc" containerID="de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239" exitCode=0
Jul 17 06:36:10.992960 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:10.992932    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" event={"ID":"785df468-28af-4515-b0f0-5c97f5f588dc","Type":"ContainerDied","Data":"de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239"}
Jul 17 06:36:10.993466 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:10.993382    2558 scope.go:117] "RemoveContainer" containerID="de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239"
Jul 17 06:36:10.994460 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:10.994435    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:36:10.994549 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:10.994534    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:36:10.994596 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:10.994582    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:36:12.001190 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:12.001146    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" event={"ID":"785df468-28af-4515-b0f0-5c97f5f588dc","Type":"ContainerStarted","Data":"ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7"}
Jul 17 06:36:12.818854 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:12.818815    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:36:12.819136 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:12.819113    2558 patch_prober.go:28] interesting pod/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Jul 17 06:36:12.819219 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:12.819164    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Jul 17 06:36:13.818688 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:13.818643    2558 patch_prober.go:28] interesting pod/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Jul 17 06:36:13.819077 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:13.818708    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Jul 17 06:36:14.818773 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:14.818732    2558 patch_prober.go:28] interesting pod/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Jul 17 06:36:14.819198 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:14.818789    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Jul 17 06:36:15.818628 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:15.818586    2558 patch_prober.go:28] interesting pod/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Jul 17 06:36:15.818801 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:15.818659    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Jul 17 06:36:16.855204 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:16.855159    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:36:16.855752 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:16.855498    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:36:16.856590 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:16.856539    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:36:17.858116 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:17.857662    2558 patch_prober.go:28] interesting pod/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": read tcp 10.134.0.2:48168->10.134.0.96:15021: read: connection reset by peer" start-of-body=
Jul 17 06:36:17.858116 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:17.857724    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": read tcp 10.134.0.2:48168->10.134.0.96:15021: read: connection reset by peer"
Jul 17 06:36:18.041654 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:18.041565    2558 generic.go:356] "Generic (PLEG): container finished" podID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerID="d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e" exitCode=0
Jul 17 06:36:18.041818 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:18.041649    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" event={"ID":"414677a6-4e6e-435b-8afc-c5696f6d8393","Type":"ContainerDied","Data":"d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e"}
Jul 17 06:36:18.042169 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:18.042151    2558 scope.go:117] "RemoveContainer" containerID="d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e"
Jul 17 06:36:18.043258 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:18.043225    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:36:18.043380 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:18.043292    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:36:18.043380 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:18.043340    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 06:36:18.274534 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:18.274504    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:36:18.280729 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:18.280700    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:36:18.281734 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:18.281713    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:36:18.292421 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:18.292393    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:36:19.052783 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:19.052735    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" event={"ID":"414677a6-4e6e-435b-8afc-c5696f6d8393","Type":"ContainerStarted","Data":"1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd"}
Jul 17 06:36:19.196234 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:19.196134    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:19.196409 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:19.196386    2558 patch_prober.go:28] interesting pod/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Jul 17 06:36:19.196465 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:19.196434    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Jul 17 06:36:20.196595 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:20.196554    2558 patch_prober.go:28] interesting pod/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Jul 17 06:36:20.197088 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:20.196656    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Jul 17 06:36:21.196187 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:21.196150    2558 patch_prober.go:28] interesting pod/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Jul 17 06:36:21.196385 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:21.196204    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Jul 17 06:36:22.196717 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:22.196680    2558 patch_prober.go:28] interesting pod/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Jul 17 06:36:22.197191 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:22.196744    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Jul 17 06:36:23.196404 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:23.196361    2558 patch_prober.go:28] interesting pod/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Jul 17 06:36:23.196632 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:23.196433    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Jul 17 06:36:24.200558 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:24.200528    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:24.201136 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:24.200925    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:24.201522 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:24.201496    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:36:35.996810 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:35.996771    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"]
Jul 17 06:36:35.997439 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:35.997418    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e0bce2f-ccc1-4b87-93a3-4bb87565a303" containerName="istio-proxy"
Jul 17 06:36:35.999577 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:35.999552    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.002150 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.002133    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"openshift-service-ca.crt\""
Jul 17 06:36:36.003296 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.003275    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"auth-test-model-346c23-kserve-self-signed-certs\""
Jul 17 06:36:36.003397 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.003275    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"kube-root-ca.crt\""
Jul 17 06:36:36.003397 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.003278    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"default-dockercfg-djgv9\""
Jul 17 06:36:36.009026 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.009006    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"]
Jul 17 06:36:36.100916 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.100890    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-model-cache\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.100916 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.100926    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-home\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.101118 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.100952    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tmp-dir\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.101118 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.100977    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tls-certs\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.101118 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.101027    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-dshm\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.101118 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.101101    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng5gs\" (UniqueName: \"kubernetes.io/projected/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-kube-api-access-ng5gs\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.202429 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.202387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-model-cache\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.202429 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.202438    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-home\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.202680 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.202464    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tmp-dir\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.202680 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.202527    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tls-certs\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.202680 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.202563    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-dshm\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.202680 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.202630    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng5gs\" (UniqueName: \"kubernetes.io/projected/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-kube-api-access-ng5gs\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.202899 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.202833    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-model-cache\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.202899 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.202854    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-home\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.203013 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.202920    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tmp-dir\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.205046 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.205026    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-dshm\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.205347 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.205325    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tls-certs\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.212657 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.212633    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng5gs\" (UniqueName: \"kubernetes.io/projected/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-kube-api-access-ng5gs\") pod \"auth-test-model-346c23-kserve-6657c6f7f-ls62k\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") " pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.309945 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.309860    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:36.457328 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:36.457299    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"]
Jul 17 06:36:36.460549 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:36:36.460520    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f9a1edf_f5ce_4bab_8339_4ead64cff38a.slice/crio-28ca3e45eaf50bee950513d7cb1f7b5b059e7686fe1ecff4483a0b30077fc88c WatchSource:0}: Error finding container 28ca3e45eaf50bee950513d7cb1f7b5b059e7686fe1ecff4483a0b30077fc88c: Status 404 returned error can't find the container with id 28ca3e45eaf50bee950513d7cb1f7b5b059e7686fe1ecff4483a0b30077fc88c
Jul 17 06:36:37.149797 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:37.149758    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k" event={"ID":"6f9a1edf-f5ce-4bab-8339-4ead64cff38a","Type":"ContainerStarted","Data":"f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f"}
Jul 17 06:36:37.150250 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:37.149803    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k" event={"ID":"6f9a1edf-f5ce-4bab-8339-4ead64cff38a","Type":"ContainerStarted","Data":"28ca3e45eaf50bee950513d7cb1f7b5b059e7686fe1ecff4483a0b30077fc88c"}
Jul 17 06:36:37.168298 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:37.168248    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k" podStartSLOduration=2.168232898 podStartE2EDuration="2.168232898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:36:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:36:37.166387006 +0000 UTC m=+3319.608325607" watchObservedRunningTime="2026-07-17 06:36:37.168232898 +0000 UTC m=+3319.610171474"
Jul 17 06:36:46.310867 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:46.310783    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:46.310867 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:46.310831    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:46.323542 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:46.323514    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:36:47.220967 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:36:47.220934    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:38:42.908360 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:42.908317    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"]
Jul 17 06:38:57.934108 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:57.934061    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k" podUID="6f9a1edf-f5ce-4bab-8339-4ead64cff38a" containerName="main" containerID="cri-o://f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f" gracePeriod=45
Jul 17 06:38:58.203422 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.203394    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:38:58.302261 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.302221    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tls-certs\") pod \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") "
Jul 17 06:38:58.302782 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.302283    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-dshm\") pod \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") "
Jul 17 06:38:58.302782 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.302341    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng5gs\" (UniqueName: \"kubernetes.io/projected/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-kube-api-access-ng5gs\") pod \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") "
Jul 17 06:38:58.302782 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.302377    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-home\") pod \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") "
Jul 17 06:38:58.302782 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.302414    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tmp-dir\") pod \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") "
Jul 17 06:38:58.302782 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.302571    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-model-cache\") pod \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\" (UID: \"6f9a1edf-f5ce-4bab-8339-4ead64cff38a\") "
Jul 17 06:38:58.302782 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.302681    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-home" (OuterVolumeSpecName: "home") pod "6f9a1edf-f5ce-4bab-8339-4ead64cff38a" (UID: "6f9a1edf-f5ce-4bab-8339-4ead64cff38a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:38:58.302782 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.302712    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f9a1edf-f5ce-4bab-8339-4ead64cff38a" (UID: "6f9a1edf-f5ce-4bab-8339-4ead64cff38a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:38:58.302782 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.302781    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f9a1edf-f5ce-4bab-8339-4ead64cff38a" (UID: "6f9a1edf-f5ce-4bab-8339-4ead64cff38a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:38:58.303141 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.303022    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-home\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:38:58.303141 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.303042    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tmp-dir\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:38:58.303141 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.303059    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-model-cache\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:38:58.304916 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.304894    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-dshm" (OuterVolumeSpecName: "dshm") pod "6f9a1edf-f5ce-4bab-8339-4ead64cff38a" (UID: "6f9a1edf-f5ce-4bab-8339-4ead64cff38a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:38:58.304994 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.304926    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f9a1edf-f5ce-4bab-8339-4ead64cff38a" (UID: "6f9a1edf-f5ce-4bab-8339-4ead64cff38a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 06:38:58.305038 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.304987    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-kube-api-access-ng5gs" (OuterVolumeSpecName: "kube-api-access-ng5gs") pod "6f9a1edf-f5ce-4bab-8339-4ead64cff38a" (UID: "6f9a1edf-f5ce-4bab-8339-4ead64cff38a"). InnerVolumeSpecName "kube-api-access-ng5gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:38:58.403799 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.403760    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng5gs\" (UniqueName: \"kubernetes.io/projected/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-kube-api-access-ng5gs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:38:58.403799 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.403801    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-tls-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:38:58.404009 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.403817    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f9a1edf-f5ce-4bab-8339-4ead64cff38a-dshm\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:38:58.941662 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.941626    2558 generic.go:356] "Generic (PLEG): container finished" podID="6f9a1edf-f5ce-4bab-8339-4ead64cff38a" containerID="f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f" exitCode=0
Jul 17 06:38:58.942111 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.941670    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"
Jul 17 06:38:58.942111 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.941722    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k" event={"ID":"6f9a1edf-f5ce-4bab-8339-4ead64cff38a","Type":"ContainerDied","Data":"f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f"}
Jul 17 06:38:58.942111 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.941768    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k" event={"ID":"6f9a1edf-f5ce-4bab-8339-4ead64cff38a","Type":"ContainerDied","Data":"28ca3e45eaf50bee950513d7cb1f7b5b059e7686fe1ecff4483a0b30077fc88c"}
Jul 17 06:38:58.942111 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.941786    2558 scope.go:117] "RemoveContainer" containerID="f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f"
Jul 17 06:38:58.952410 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.952384    2558 scope.go:117] "RemoveContainer" containerID="f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f"
Jul 17 06:38:58.952759 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:38:58.952737    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f\": container with ID starting with f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f not found: ID does not exist" containerID="f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f"
Jul 17 06:38:58.952827 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.952770    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f"} err="failed to get container status \"f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f\": rpc error: code = NotFound desc = could not find container \"f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f\": container with ID starting with f0b67aca68a1001098b50d929a33eb8ed86c197117dc37c1d5d1eb4d7aee9e6f not found: ID does not exist"
Jul 17 06:38:58.970597 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.970558    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"]
Jul 17 06:38:58.974979 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:38:58.974945    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/auth-test-model-346c23-kserve-6657c6f7f-ls62k"]
Jul 17 06:39:00.079217 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:00.079183    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f9a1edf-f5ce-4bab-8339-4ead64cff38a" path="/var/lib/kubelet/pods/6f9a1edf-f5ce-4bab-8339-4ead64cff38a/volumes"
Jul 17 06:39:24.685782 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.685719    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"]
Jul 17 06:39:24.686583 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.686514    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9a1edf-f5ce-4bab-8339-4ead64cff38a" containerName="main"
Jul 17 06:39:24.686583 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.686541    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9a1edf-f5ce-4bab-8339-4ead64cff38a" containerName="main"
Jul 17 06:39:24.686721 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.686676    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9a1edf-f5ce-4bab-8339-4ead64cff38a" containerName="main"
Jul 17 06:39:24.689846 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.689820    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.693572 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.693542    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"kube-root-ca.crt\""
Jul 17 06:39:24.693741 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.693542    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"default-dockercfg-djgv9\""
Jul 17 06:39:24.693741 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.693589    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"sub-test-model-346c23-kserve-self-signed-certs\""
Jul 17 06:39:24.693741 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.693589    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"openshift-service-ca.crt\""
Jul 17 06:39:24.700250 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.700217    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"]
Jul 17 06:39:24.759533 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.759497    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-model-cache\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.759533 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.759539    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-dshm\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.759798 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.759607    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-home\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.759798 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.759638    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv2m9\" (UniqueName: \"kubernetes.io/projected/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-kube-api-access-hv2m9\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.759798 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.759709    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tmp-dir\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.759798 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.759757    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tls-certs\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.860362 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.860320    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-home\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.860588 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.860374    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv2m9\" (UniqueName: \"kubernetes.io/projected/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-kube-api-access-hv2m9\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.860588 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.860404    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tmp-dir\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.860588 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.860448    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tls-certs\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.860588 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.860565    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-model-cache\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.860828 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.860597    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-dshm\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.860828 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.860746    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-home\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.860939 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.860826    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tmp-dir\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.860939 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.860890    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-model-cache\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.862958 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.862932    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-dshm\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.863272 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.863252    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tls-certs\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:24.868028 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:24.867992    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv2m9\" (UniqueName: \"kubernetes.io/projected/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-kube-api-access-hv2m9\") pod \"sub-test-model-346c23-kserve-6bbd9669c-r8r9b\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") " pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:25.001893 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:25.001783    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:25.160976 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:25.160942    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"]
Jul 17 06:39:25.162361 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:39:25.162333    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8b4f85b_a0b2_479f_88f5_fdf49119f00f.slice/crio-fc2b8d3cbd1993d96d1e076ef7f94f09c8b69b487f3cab3004b06f78c2cd6df9 WatchSource:0}: Error finding container fc2b8d3cbd1993d96d1e076ef7f94f09c8b69b487f3cab3004b06f78c2cd6df9: Status 404 returned error can't find the container with id fc2b8d3cbd1993d96d1e076ef7f94f09c8b69b487f3cab3004b06f78c2cd6df9
Jul 17 06:39:26.095714 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:26.095666    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b" event={"ID":"f8b4f85b-a0b2-479f-88f5-fdf49119f00f","Type":"ContainerStarted","Data":"ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c"}
Jul 17 06:39:26.096159 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:26.095728    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b" event={"ID":"f8b4f85b-a0b2-479f-88f5-fdf49119f00f","Type":"ContainerStarted","Data":"fc2b8d3cbd1993d96d1e076ef7f94f09c8b69b487f3cab3004b06f78c2cd6df9"}
Jul 17 06:39:26.113057 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:26.112833    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b" podStartSLOduration=2.112812864 podStartE2EDuration="2.112812864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:39:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:39:26.111741372 +0000 UTC m=+3488.553679947" watchObservedRunningTime="2026-07-17 06:39:26.112812864 +0000 UTC m=+3488.554751438"
Jul 17 06:39:35.002265 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:35.002233    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:35.002783 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:35.002277    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:35.014897 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:35.014875    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:35.154867 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:35.154841    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:39:56.891978 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:39:56.891934    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"]
Jul 17 06:40:11.913759 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:11.913716    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b" podUID="f8b4f85b-a0b2-479f-88f5-fdf49119f00f" containerName="main" containerID="cri-o://ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c" gracePeriod=45
Jul 17 06:40:12.184719 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.184687    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:40:12.250065 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.250037    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tls-certs\") pod \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") "
Jul 17 06:40:12.250201 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.250078    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv2m9\" (UniqueName: \"kubernetes.io/projected/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-kube-api-access-hv2m9\") pod \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") "
Jul 17 06:40:12.250201 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.250123    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-dshm\") pod \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") "
Jul 17 06:40:12.250201 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.250150    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-model-cache\") pod \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") "
Jul 17 06:40:12.250201 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.250184    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-home\") pod \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") "
Jul 17 06:40:12.250412 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.250235    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tmp-dir\") pod \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\" (UID: \"f8b4f85b-a0b2-479f-88f5-fdf49119f00f\") "
Jul 17 06:40:12.250502 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.250423    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-model-cache" (OuterVolumeSpecName: "model-cache") pod "f8b4f85b-a0b2-479f-88f5-fdf49119f00f" (UID: "f8b4f85b-a0b2-479f-88f5-fdf49119f00f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:40:12.250579 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.250544    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-home" (OuterVolumeSpecName: "home") pod "f8b4f85b-a0b2-479f-88f5-fdf49119f00f" (UID: "f8b4f85b-a0b2-479f-88f5-fdf49119f00f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:40:12.250681 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.250654    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f8b4f85b-a0b2-479f-88f5-fdf49119f00f" (UID: "f8b4f85b-a0b2-479f-88f5-fdf49119f00f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:40:12.250747 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.250711    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-model-cache\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:40:12.250747 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.250732    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-home\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:40:12.252409 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.252378    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-dshm" (OuterVolumeSpecName: "dshm") pod "f8b4f85b-a0b2-479f-88f5-fdf49119f00f" (UID: "f8b4f85b-a0b2-479f-88f5-fdf49119f00f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:40:12.252547 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.252409    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f8b4f85b-a0b2-479f-88f5-fdf49119f00f" (UID: "f8b4f85b-a0b2-479f-88f5-fdf49119f00f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 06:40:12.252547 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.252464    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-kube-api-access-hv2m9" (OuterVolumeSpecName: "kube-api-access-hv2m9") pod "f8b4f85b-a0b2-479f-88f5-fdf49119f00f" (UID: "f8b4f85b-a0b2-479f-88f5-fdf49119f00f"). InnerVolumeSpecName "kube-api-access-hv2m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:40:12.347044 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.347007    2558 generic.go:356] "Generic (PLEG): container finished" podID="f8b4f85b-a0b2-479f-88f5-fdf49119f00f" containerID="ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c" exitCode=0
Jul 17 06:40:12.347173 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.347060    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"
Jul 17 06:40:12.347173 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.347092    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b" event={"ID":"f8b4f85b-a0b2-479f-88f5-fdf49119f00f","Type":"ContainerDied","Data":"ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c"}
Jul 17 06:40:12.347173 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.347140    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b" event={"ID":"f8b4f85b-a0b2-479f-88f5-fdf49119f00f","Type":"ContainerDied","Data":"fc2b8d3cbd1993d96d1e076ef7f94f09c8b69b487f3cab3004b06f78c2cd6df9"}
Jul 17 06:40:12.347173 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.347162    2558 scope.go:117] "RemoveContainer" containerID="ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c"
Jul 17 06:40:12.351272 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.351251    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tls-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:40:12.351272 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.351273    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv2m9\" (UniqueName: \"kubernetes.io/projected/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-kube-api-access-hv2m9\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:40:12.351457 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.351283    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-dshm\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:40:12.351457 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.351292    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8b4f85b-a0b2-479f-88f5-fdf49119f00f-tmp-dir\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:40:12.357030 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.357015    2558 scope.go:117] "RemoveContainer" containerID="ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c"
Jul 17 06:40:12.357303 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:40:12.357285    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c\": container with ID starting with ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c not found: ID does not exist" containerID="ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c"
Jul 17 06:40:12.357352 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.357311    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c"} err="failed to get container status \"ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c\": rpc error: code = NotFound desc = could not find container \"ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c\": container with ID starting with ffe59ad0076e1511a40699277b98a17dcbc20f037bd44da40ae4815453389f9c not found: ID does not exist"
Jul 17 06:40:12.386111 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.386090    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"]
Jul 17 06:40:12.391162 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:12.391142    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/sub-test-model-346c23-kserve-6bbd9669c-r8r9b"]
Jul 17 06:40:14.079312 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:14.079278    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8b4f85b-a0b2-479f-88f5-fdf49119f00f" path="/var/lib/kubelet/pods/f8b4f85b-a0b2-479f-88f5-fdf49119f00f/volumes"
Jul 17 06:40:20.653622 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:20.653586    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"]
Jul 17 06:40:20.654016 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:20.654003    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8b4f85b-a0b2-479f-88f5-fdf49119f00f" containerName="main"
Jul 17 06:40:20.654061 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:20.654018    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8b4f85b-a0b2-479f-88f5-fdf49119f00f" containerName="main"
Jul 17 06:40:20.654097 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:20.654087    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8b4f85b-a0b2-479f-88f5-fdf49119f00f" containerName="main"
Jul 17 06:40:20.658589 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:20.658566    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"
Jul 17 06:40:20.661507 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:20.661416    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtfhw\""
Jul 17 06:40:20.664954 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:20.664726    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"]
Jul 17 06:40:20.825830 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:20.825783    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd17f6fd-d2fd-4d2c-b591-3b2e4d188764-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8\" (UID: \"bd17f6fd-d2fd-4d2c-b591-3b2e4d188764\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"
Jul 17 06:40:20.926582 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:20.926496    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd17f6fd-d2fd-4d2c-b591-3b2e4d188764-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8\" (UID: \"bd17f6fd-d2fd-4d2c-b591-3b2e4d188764\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"
Jul 17 06:40:20.927152 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:20.927128    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd17f6fd-d2fd-4d2c-b591-3b2e4d188764-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8\" (UID: \"bd17f6fd-d2fd-4d2c-b591-3b2e4d188764\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"
Jul 17 06:40:20.971805 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:20.971779    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"
Jul 17 06:40:21.114587 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:21.114561    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"]
Jul 17 06:40:21.116932 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:40:21.116907    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd17f6fd_d2fd_4d2c_b591_3b2e4d188764.slice/crio-2369ce93d2c849d76c3a63593f3267ea7198a26dc4ccda0410edd8d447bea707 WatchSource:0}: Error finding container 2369ce93d2c849d76c3a63593f3267ea7198a26dc4ccda0410edd8d447bea707: Status 404 returned error can't find the container with id 2369ce93d2c849d76c3a63593f3267ea7198a26dc4ccda0410edd8d447bea707
Jul 17 06:40:21.392947 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:21.392922    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8" event={"ID":"bd17f6fd-d2fd-4d2c-b591-3b2e4d188764","Type":"ContainerStarted","Data":"2369ce93d2c849d76c3a63593f3267ea7198a26dc4ccda0410edd8d447bea707"}
Jul 17 06:40:22.400059 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:22.400025    2558 generic.go:356] "Generic (PLEG): container finished" podID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerID="f14db739d5ecd005e2bc3766ea358a6b7428f9f323b69d9cb872262ae3f28fb9" exitCode=6
Jul 17 06:40:22.400448 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:22.400110    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8" event={"ID":"bd17f6fd-d2fd-4d2c-b591-3b2e4d188764","Type":"ContainerDied","Data":"f14db739d5ecd005e2bc3766ea358a6b7428f9f323b69d9cb872262ae3f28fb9"}
Jul 17 06:40:22.400448 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:22.400305    2558 scope.go:117] "RemoveContainer" containerID="f14db739d5ecd005e2bc3766ea358a6b7428f9f323b69d9cb872262ae3f28fb9"
Jul 17 06:40:23.408607 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:23.408510    2558 generic.go:356] "Generic (PLEG): container finished" podID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerID="d12378bc95f9f88ac8acf447c5d759352821206909380ec9e6275d3cd7e792a1" exitCode=6
Jul 17 06:40:23.408607 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:23.408588    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8" event={"ID":"bd17f6fd-d2fd-4d2c-b591-3b2e4d188764","Type":"ContainerDied","Data":"d12378bc95f9f88ac8acf447c5d759352821206909380ec9e6275d3cd7e792a1"}
Jul 17 06:40:23.409112 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:23.408632    2558 scope.go:117] "RemoveContainer" containerID="f14db739d5ecd005e2bc3766ea358a6b7428f9f323b69d9cb872262ae3f28fb9"
Jul 17 06:40:23.409112 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:23.408867    2558 scope.go:117] "RemoveContainer" containerID="d12378bc95f9f88ac8acf447c5d759352821206909380ec9e6275d3cd7e792a1"
Jul 17 06:40:23.409221 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:40:23.409146    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8_odh-ai-gateway-infra(bd17f6fd-d2fd-4d2c-b591-3b2e4d188764)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764"
Jul 17 06:40:24.416280 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:24.416242    2558 scope.go:117] "RemoveContainer" containerID="d12378bc95f9f88ac8acf447c5d759352821206909380ec9e6275d3cd7e792a1"
Jul 17 06:40:24.416781 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:40:24.416579    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8_odh-ai-gateway-infra(bd17f6fd-d2fd-4d2c-b591-3b2e4d188764)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764"
Jul 17 06:40:37.589566 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.589525    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"]
Jul 17 06:40:37.595060 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.595030    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.600085 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.599631    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"openshift-service-ca.crt\""
Jul 17 06:40:37.600085 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.599933    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"rate-test-model-346c23-kserve-self-signed-certs\""
Jul 17 06:40:37.600085 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.599955    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"kube-root-ca.crt\""
Jul 17 06:40:37.600346 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.600166    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-346c23\"/\"default-dockercfg-djgv9\""
Jul 17 06:40:37.603194 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.603161    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"]
Jul 17 06:40:37.688398 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.688361    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-home\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.688603 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.688404    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m57qw\" (UniqueName: \"kubernetes.io/projected/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-kube-api-access-m57qw\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.688603 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.688435    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-dshm\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.688603 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.688461    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-model-cache\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.688603 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.688562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tls-certs\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.688807 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.688620    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tmp-dir\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.789845 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.789814    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-home\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.789845 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.789847    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m57qw\" (UniqueName: \"kubernetes.io/projected/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-kube-api-access-m57qw\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.790091 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.789867    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-dshm\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.790091 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.789889    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-model-cache\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.790091 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.789945    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tls-certs\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.790091 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.789994    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tmp-dir\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.790329 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.790301    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-home\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.790421 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.790393    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-model-cache\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.790421 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.790415    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tmp-dir\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.792301 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.792283    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-dshm\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.792733 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.792713    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tls-certs\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.797901 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.797883    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m57qw\" (UniqueName: \"kubernetes.io/projected/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-kube-api-access-m57qw\") pod \"rate-test-model-346c23-kserve-568c5f7659-6j44w\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") " pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:37.907052 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:37.906964    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:38.047381 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:38.047359    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"]
Jul 17 06:40:38.049529 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:40:38.049498    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f36db3c_2e3f_4af9_aeab_11d6bb91707a.slice/crio-a5d3bd1610711b8aca0f91757b358106a86bc24baab0a595018eb6ea24eb1e5a WatchSource:0}: Error finding container a5d3bd1610711b8aca0f91757b358106a86bc24baab0a595018eb6ea24eb1e5a: Status 404 returned error can't find the container with id a5d3bd1610711b8aca0f91757b358106a86bc24baab0a595018eb6ea24eb1e5a
Jul 17 06:40:38.074904 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:38.074884    2558 scope.go:117] "RemoveContainer" containerID="d12378bc95f9f88ac8acf447c5d759352821206909380ec9e6275d3cd7e792a1"
Jul 17 06:40:38.494062 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:38.493976    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8" event={"ID":"bd17f6fd-d2fd-4d2c-b591-3b2e4d188764","Type":"ContainerStarted","Data":"798c5b04bc30b7e89c59a655b08f9c1e2d4fb8bb09ae8c346a17ec4f2fab282a"}
Jul 17 06:40:38.495528 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:38.495505    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w" event={"ID":"4f36db3c-2e3f-4af9-aeab-11d6bb91707a","Type":"ContainerStarted","Data":"8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6"}
Jul 17 06:40:38.495644 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:38.495537    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w" event={"ID":"4f36db3c-2e3f-4af9-aeab-11d6bb91707a","Type":"ContainerStarted","Data":"a5d3bd1610711b8aca0f91757b358106a86bc24baab0a595018eb6ea24eb1e5a"}
Jul 17 06:40:38.513040 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:38.512995    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8" podStartSLOduration=18.512980436 podStartE2EDuration="18.512980436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:40:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:40:38.510520064 +0000 UTC m=+3560.952458662" watchObservedRunningTime="2026-07-17 06:40:38.512980436 +0000 UTC m=+3560.954919011"
Jul 17 06:40:38.527576 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:38.527530    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w" podStartSLOduration=1.527515453 podStartE2EDuration="1.527515453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:40:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:40:38.526843141 +0000 UTC m=+3560.968781717" watchObservedRunningTime="2026-07-17 06:40:38.527515453 +0000 UTC m=+3560.969454028"
Jul 17 06:40:39.502406 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:39.502370    2558 generic.go:356] "Generic (PLEG): container finished" podID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerID="798c5b04bc30b7e89c59a655b08f9c1e2d4fb8bb09ae8c346a17ec4f2fab282a" exitCode=6
Jul 17 06:40:39.502830 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:39.502444    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8" event={"ID":"bd17f6fd-d2fd-4d2c-b591-3b2e4d188764","Type":"ContainerDied","Data":"798c5b04bc30b7e89c59a655b08f9c1e2d4fb8bb09ae8c346a17ec4f2fab282a"}
Jul 17 06:40:39.502830 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:39.502521    2558 scope.go:117] "RemoveContainer" containerID="d12378bc95f9f88ac8acf447c5d759352821206909380ec9e6275d3cd7e792a1"
Jul 17 06:40:39.502927 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:39.502893    2558 scope.go:117] "RemoveContainer" containerID="798c5b04bc30b7e89c59a655b08f9c1e2d4fb8bb09ae8c346a17ec4f2fab282a"
Jul 17 06:40:39.503150 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:40:39.503126    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8_odh-ai-gateway-infra(bd17f6fd-d2fd-4d2c-b591-3b2e4d188764)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764"
Jul 17 06:40:39.526808 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:39.526787    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"]
Jul 17 06:40:40.661544 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:40.661509    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"
Jul 17 06:40:40.717374 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:40.717346    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd17f6fd-d2fd-4d2c-b591-3b2e4d188764-maas-api-service-ca\") pod \"bd17f6fd-d2fd-4d2c-b591-3b2e4d188764\" (UID: \"bd17f6fd-d2fd-4d2c-b591-3b2e4d188764\") "
Jul 17 06:40:40.717714 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:40.717694    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd17f6fd-d2fd-4d2c-b591-3b2e4d188764-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" (UID: "bd17f6fd-d2fd-4d2c-b591-3b2e4d188764"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:40:40.818319 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:40.818250    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd17f6fd-d2fd-4d2c-b591-3b2e4d188764-maas-api-service-ca\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:40:41.515688 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:41.515660    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"
Jul 17 06:40:41.515688 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:41.515672    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8" event={"ID":"bd17f6fd-d2fd-4d2c-b591-3b2e4d188764","Type":"ContainerDied","Data":"2369ce93d2c849d76c3a63593f3267ea7198a26dc4ccda0410edd8d447bea707"}
Jul 17 06:40:41.515888 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:41.515731    2558 scope.go:117] "RemoveContainer" containerID="798c5b04bc30b7e89c59a655b08f9c1e2d4fb8bb09ae8c346a17ec4f2fab282a"
Jul 17 06:40:41.544834 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:41.544810    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"]
Jul 17 06:40:41.552923 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:41.552899    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51e238b9-ncdh8"]
Jul 17 06:40:42.078840 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:42.078811    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" path="/var/lib/kubelet/pods/bd17f6fd-d2fd-4d2c-b591-3b2e4d188764/volumes"
Jul 17 06:40:47.908061 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:47.908033    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:47.908407 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:47.908080    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:47.920541 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:47.920521    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:40:48.568464 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:40:48.568429    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:41:18.315950 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:18.315920    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:41:18.321778 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:18.321758    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:41:18.325990 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:18.325969    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:41:18.332586 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:18.332566    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fj8r8_56e41793-4279-46d8-84a0-ad435e812001/ovn-acl-logging/0.log"
Jul 17 06:41:26.434128 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:26.434061    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"]
Jul 17 06:41:41.454649 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.454576    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w" podUID="4f36db3c-2e3f-4af9-aeab-11d6bb91707a" containerName="main" containerID="cri-o://8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6" gracePeriod=45
Jul 17 06:41:41.708868 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.708811    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:41:41.817009 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.816976    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-dshm\") pod \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") "
Jul 17 06:41:41.817187 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.817069    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m57qw\" (UniqueName: \"kubernetes.io/projected/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-kube-api-access-m57qw\") pod \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") "
Jul 17 06:41:41.817187 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.817095    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-model-cache\") pod \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") "
Jul 17 06:41:41.817187 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.817130    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tls-certs\") pod \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") "
Jul 17 06:41:41.817187 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.817154    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tmp-dir\") pod \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") "
Jul 17 06:41:41.817423 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.817305    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-home\") pod \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\" (UID: \"4f36db3c-2e3f-4af9-aeab-11d6bb91707a\") "
Jul 17 06:41:41.817423 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.817380    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f36db3c-2e3f-4af9-aeab-11d6bb91707a" (UID: "4f36db3c-2e3f-4af9-aeab-11d6bb91707a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:41:41.817423 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.817383    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f36db3c-2e3f-4af9-aeab-11d6bb91707a" (UID: "4f36db3c-2e3f-4af9-aeab-11d6bb91707a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:41:41.817624 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.817587    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-home" (OuterVolumeSpecName: "home") pod "4f36db3c-2e3f-4af9-aeab-11d6bb91707a" (UID: "4f36db3c-2e3f-4af9-aeab-11d6bb91707a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:41:41.817757 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.817737    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-home\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:41:41.817815 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.817764    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-model-cache\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:41:41.817815 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.817782    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tmp-dir\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:41:41.819520 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.819487    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f36db3c-2e3f-4af9-aeab-11d6bb91707a" (UID: "4f36db3c-2e3f-4af9-aeab-11d6bb91707a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 06:41:41.819665 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.819543    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-kube-api-access-m57qw" (OuterVolumeSpecName: "kube-api-access-m57qw") pod "4f36db3c-2e3f-4af9-aeab-11d6bb91707a" (UID: "4f36db3c-2e3f-4af9-aeab-11d6bb91707a"). InnerVolumeSpecName "kube-api-access-m57qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:41:41.819736 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.819700    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-dshm" (OuterVolumeSpecName: "dshm") pod "4f36db3c-2e3f-4af9-aeab-11d6bb91707a" (UID: "4f36db3c-2e3f-4af9-aeab-11d6bb91707a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:41:41.855883 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.855856    2558 generic.go:356] "Generic (PLEG): container finished" podID="4f36db3c-2e3f-4af9-aeab-11d6bb91707a" containerID="8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6" exitCode=0
Jul 17 06:41:41.856005 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.855899    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w" event={"ID":"4f36db3c-2e3f-4af9-aeab-11d6bb91707a","Type":"ContainerDied","Data":"8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6"}
Jul 17 06:41:41.856005 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.855903    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"
Jul 17 06:41:41.856005 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.855930    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w" event={"ID":"4f36db3c-2e3f-4af9-aeab-11d6bb91707a","Type":"ContainerDied","Data":"a5d3bd1610711b8aca0f91757b358106a86bc24baab0a595018eb6ea24eb1e5a"}
Jul 17 06:41:41.856005 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.855950    2558 scope.go:117] "RemoveContainer" containerID="8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6"
Jul 17 06:41:41.865406 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.865383    2558 scope.go:117] "RemoveContainer" containerID="8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6"
Jul 17 06:41:41.865689 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:41:41.865671    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6\": container with ID starting with 8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6 not found: ID does not exist" containerID="8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6"
Jul 17 06:41:41.865756 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.865699    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6"} err="failed to get container status \"8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6\": rpc error: code = NotFound desc = could not find container \"8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6\": container with ID starting with 8164839bd93c4bf290e98b6ceef0ea8290aab215ceafb8e70b6617ce30d838c6 not found: ID does not exist"
Jul 17 06:41:41.878825 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.878802    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"]
Jul 17 06:41:41.883099 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.883078    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-346c23/rate-test-model-346c23-kserve-568c5f7659-6j44w"]
Jul 17 06:41:41.918796 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.918772    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-dshm\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:41:41.918796 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.918794    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m57qw\" (UniqueName: \"kubernetes.io/projected/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-kube-api-access-m57qw\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:41:41.918919 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:41.918804    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f36db3c-2e3f-4af9-aeab-11d6bb91707a-tls-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:41:42.079181 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:41:42.079098    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f36db3c-2e3f-4af9-aeab-11d6bb91707a" path="/var/lib/kubelet/pods/4f36db3c-2e3f-4af9-aeab-11d6bb91707a/volumes"
Jul 17 06:42:07.462532 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:07.462494    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"]
Jul 17 06:42:07.463109 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:07.462706    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy" containerID="cri-o://ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7" gracePeriod=30
Jul 17 06:42:12.715218 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.715188    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:42:12.810970 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.810911    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-certs\") pod \"785df468-28af-4515-b0f0-5c97f5f588dc\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") "
Jul 17 06:42:12.810970 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.810947    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/785df468-28af-4515-b0f0-5c97f5f588dc-istio-podinfo\") pod \"785df468-28af-4515-b0f0-5c97f5f588dc\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") "
Jul 17 06:42:12.811164 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.810973    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-istio-token\") pod \"785df468-28af-4515-b0f0-5c97f5f588dc\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") "
Jul 17 06:42:12.811164 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811010    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-socket\") pod \"785df468-28af-4515-b0f0-5c97f5f588dc\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") "
Jul 17 06:42:12.811164 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811032    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-data\") pod \"785df468-28af-4515-b0f0-5c97f5f588dc\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") "
Jul 17 06:42:12.811164 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811075    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92cws\" (UniqueName: \"kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-kube-api-access-92cws\") pod \"785df468-28af-4515-b0f0-5c97f5f588dc\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") "
Jul 17 06:42:12.811164 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811107    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-credential-socket\") pod \"785df468-28af-4515-b0f0-5c97f5f588dc\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") "
Jul 17 06:42:12.811164 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811140    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-envoy\") pod \"785df468-28af-4515-b0f0-5c97f5f588dc\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") "
Jul 17 06:42:12.811560 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811171    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "785df468-28af-4515-b0f0-5c97f5f588dc" (UID: "785df468-28af-4515-b0f0-5c97f5f588dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:42:12.811560 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811189    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/785df468-28af-4515-b0f0-5c97f5f588dc-istiod-ca-cert\") pod \"785df468-28af-4515-b0f0-5c97f5f588dc\" (UID: \"785df468-28af-4515-b0f0-5c97f5f588dc\") "
Jul 17 06:42:12.811560 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811372    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "785df468-28af-4515-b0f0-5c97f5f588dc" (UID: "785df468-28af-4515-b0f0-5c97f5f588dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:42:12.811560 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811383    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "785df468-28af-4515-b0f0-5c97f5f588dc" (UID: "785df468-28af-4515-b0f0-5c97f5f588dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:42:12.811560 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811464    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:12.811560 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811513    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:12.811560 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811527    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:12.812089 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.811675    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/785df468-28af-4515-b0f0-5c97f5f588dc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "785df468-28af-4515-b0f0-5c97f5f588dc" (UID: "785df468-28af-4515-b0f0-5c97f5f588dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:42:12.812321 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.812294    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-data" (OuterVolumeSpecName: "istio-data") pod "785df468-28af-4515-b0f0-5c97f5f588dc" (UID: "785df468-28af-4515-b0f0-5c97f5f588dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:42:12.813241 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.813219    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/785df468-28af-4515-b0f0-5c97f5f588dc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "785df468-28af-4515-b0f0-5c97f5f588dc" (UID: "785df468-28af-4515-b0f0-5c97f5f588dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:42:12.813433 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.813401    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-kube-api-access-92cws" (OuterVolumeSpecName: "kube-api-access-92cws") pod "785df468-28af-4515-b0f0-5c97f5f588dc" (UID: "785df468-28af-4515-b0f0-5c97f5f588dc"). InnerVolumeSpecName "kube-api-access-92cws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:42:12.813546 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.813447    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-istio-token" (OuterVolumeSpecName: "istio-token") pod "785df468-28af-4515-b0f0-5c97f5f588dc" (UID: "785df468-28af-4515-b0f0-5c97f5f588dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:42:12.813798 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.813768    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "785df468-28af-4515-b0f0-5c97f5f588dc" (UID: "785df468-28af-4515-b0f0-5c97f5f588dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:42:12.912696 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.912671    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:12.912696 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.912695    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92cws\" (UniqueName: \"kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-kube-api-access-92cws\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:12.912841 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.912711    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/785df468-28af-4515-b0f0-5c97f5f588dc-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:12.912841 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.912726    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/785df468-28af-4515-b0f0-5c97f5f588dc-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:12.912841 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.912739    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/785df468-28af-4515-b0f0-5c97f5f588dc-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:12.912841 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:12.912750    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/785df468-28af-4515-b0f0-5c97f5f588dc-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:13.024758 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.024730    2558 generic.go:356] "Generic (PLEG): container finished" podID="785df468-28af-4515-b0f0-5c97f5f588dc" containerID="ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7" exitCode=0
Jul 17 06:42:13.024872 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.024780    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"
Jul 17 06:42:13.024872 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.024815    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" event={"ID":"785df468-28af-4515-b0f0-5c97f5f588dc","Type":"ContainerDied","Data":"ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7"}
Jul 17 06:42:13.024991 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.024887    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb" event={"ID":"785df468-28af-4515-b0f0-5c97f5f588dc","Type":"ContainerDied","Data":"22f5dabc87e98e9fc75a1f77e3f4d0450b9a2a2358aa52338a5c224327ea1238"}
Jul 17 06:42:13.024991 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.024922    2558 scope.go:117] "RemoveContainer" containerID="ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7"
Jul 17 06:42:13.035185 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.035171    2558 scope.go:117] "RemoveContainer" containerID="de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239"
Jul 17 06:42:13.043726 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.043707    2558 scope.go:117] "RemoveContainer" containerID="ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7"
Jul 17 06:42:13.043965 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:42:13.043950    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7\": container with ID starting with ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7 not found: ID does not exist" containerID="ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7"
Jul 17 06:42:13.044019 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.043971    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7"} err="failed to get container status \"ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7\": rpc error: code = NotFound desc = could not find container \"ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7\": container with ID starting with ea24b62bfb56483dfd0812787b108dcbcdaaee842ecfc6c54fe7d053ec306ff7 not found: ID does not exist"
Jul 17 06:42:13.044019 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.043986    2558 scope.go:117] "RemoveContainer" containerID="de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239"
Jul 17 06:42:13.044199 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:42:13.044182    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239\": container with ID starting with de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239 not found: ID does not exist" containerID="de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239"
Jul 17 06:42:13.044253 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.044210    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239"} err="failed to get container status \"de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239\": rpc error: code = NotFound desc = could not find container \"de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239\": container with ID starting with de8bb7eba97fe71881cea51cf869ba4d2056ea3d65485129399b949654def239 not found: ID does not exist"
Jul 17 06:42:13.051620 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.051599    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"]
Jul 17 06:42:13.057176 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:13.057156    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-ff4257-openshift-default-788c6799f-7p9nb"]
Jul 17 06:42:14.078120 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:14.078086    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" path="/var/lib/kubelet/pods/785df468-28af-4515-b0f0-5c97f5f588dc/volumes"
Jul 17 06:42:30.519972 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:30.519896    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"]
Jul 17 06:42:30.520560 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:30.520118    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy" containerID="cri-o://1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd" gracePeriod=30
Jul 17 06:42:35.778466 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.778438    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:42:35.918702 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.918667    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-socket\") pod \"414677a6-4e6e-435b-8afc-c5696f6d8393\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") "
Jul 17 06:42:35.918892 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.918723    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmxgr\" (UniqueName: \"kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-kube-api-access-xmxgr\") pod \"414677a6-4e6e-435b-8afc-c5696f6d8393\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") "
Jul 17 06:42:35.918892 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.918758    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-podinfo\") pod \"414677a6-4e6e-435b-8afc-c5696f6d8393\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") "
Jul 17 06:42:35.918892 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.918781    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-certs\") pod \"414677a6-4e6e-435b-8afc-c5696f6d8393\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") "
Jul 17 06:42:35.918892 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.918809    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-token\") pod \"414677a6-4e6e-435b-8afc-c5696f6d8393\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") "
Jul 17 06:42:35.918892 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.918846    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414677a6-4e6e-435b-8afc-c5696f6d8393-istiod-ca-cert\") pod \"414677a6-4e6e-435b-8afc-c5696f6d8393\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") "
Jul 17 06:42:35.919165 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.918906    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-credential-socket\") pod \"414677a6-4e6e-435b-8afc-c5696f6d8393\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") "
Jul 17 06:42:35.919165 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.918942    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-envoy\") pod \"414677a6-4e6e-435b-8afc-c5696f6d8393\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") "
Jul 17 06:42:35.919165 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.918946    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "414677a6-4e6e-435b-8afc-c5696f6d8393" (UID: "414677a6-4e6e-435b-8afc-c5696f6d8393"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:42:35.919165 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.919021    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-data\") pod \"414677a6-4e6e-435b-8afc-c5696f6d8393\" (UID: \"414677a6-4e6e-435b-8afc-c5696f6d8393\") "
Jul 17 06:42:35.919165 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.919104    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "414677a6-4e6e-435b-8afc-c5696f6d8393" (UID: "414677a6-4e6e-435b-8afc-c5696f6d8393"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:42:35.919439 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.919198    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "414677a6-4e6e-435b-8afc-c5696f6d8393" (UID: "414677a6-4e6e-435b-8afc-c5696f6d8393"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:42:35.919439 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.919280    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/414677a6-4e6e-435b-8afc-c5696f6d8393-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "414677a6-4e6e-435b-8afc-c5696f6d8393" (UID: "414677a6-4e6e-435b-8afc-c5696f6d8393"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 06:42:35.919580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.919451    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-certs\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:35.919580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.919493    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-credential-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:35.919580 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.919510    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-workload-socket\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:35.919941 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.919920    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-data" (OuterVolumeSpecName: "istio-data") pod "414677a6-4e6e-435b-8afc-c5696f6d8393" (UID: "414677a6-4e6e-435b-8afc-c5696f6d8393"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:42:35.921227 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.921205    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "414677a6-4e6e-435b-8afc-c5696f6d8393" (UID: "414677a6-4e6e-435b-8afc-c5696f6d8393"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 06:42:35.921415 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.921398    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "414677a6-4e6e-435b-8afc-c5696f6d8393" (UID: "414677a6-4e6e-435b-8afc-c5696f6d8393"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 06:42:35.921662 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.921635    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-kube-api-access-xmxgr" (OuterVolumeSpecName: "kube-api-access-xmxgr") pod "414677a6-4e6e-435b-8afc-c5696f6d8393" (UID: "414677a6-4e6e-435b-8afc-c5696f6d8393"). InnerVolumeSpecName "kube-api-access-xmxgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:42:35.921662 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:35.921635    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-token" (OuterVolumeSpecName: "istio-token") pod "414677a6-4e6e-435b-8afc-c5696f6d8393" (UID: "414677a6-4e6e-435b-8afc-c5696f6d8393"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 06:42:36.020684 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.020655    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414677a6-4e6e-435b-8afc-c5696f6d8393-istiod-ca-cert\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:36.020684 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.020680    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-envoy\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:36.020855 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.020691    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-data\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:36.020855 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.020703    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmxgr\" (UniqueName: \"kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-kube-api-access-xmxgr\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:36.020855 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.020715    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-podinfo\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:36.020855 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.020726    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414677a6-4e6e-435b-8afc-c5696f6d8393-istio-token\") on node \"ip-10-0-140-96.ec2.internal\" DevicePath \"\""
Jul 17 06:42:36.147517 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.147464    2558 generic.go:356] "Generic (PLEG): container finished" podID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerID="1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd" exitCode=0
Jul 17 06:42:36.147668 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.147511    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" event={"ID":"414677a6-4e6e-435b-8afc-c5696f6d8393","Type":"ContainerDied","Data":"1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd"}
Jul 17 06:42:36.147668 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.147539    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"
Jul 17 06:42:36.147668 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.147578    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv" event={"ID":"414677a6-4e6e-435b-8afc-c5696f6d8393","Type":"ContainerDied","Data":"ac2e088b9be2c354954a7f73234fdb187dbb489a8687e823424a259c889f4b4a"}
Jul 17 06:42:36.147668 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.147611    2558 scope.go:117] "RemoveContainer" containerID="1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd"
Jul 17 06:42:36.156586 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.156554    2558 scope.go:117] "RemoveContainer" containerID="d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e"
Jul 17 06:42:36.164691 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.164676    2558 scope.go:117] "RemoveContainer" containerID="1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd"
Jul 17 06:42:36.164928 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:42:36.164911    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd\": container with ID starting with 1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd not found: ID does not exist" containerID="1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd"
Jul 17 06:42:36.164977 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.164935    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd"} err="failed to get container status \"1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd\": rpc error: code = NotFound desc = could not find container \"1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd\": container with ID starting with 1a2d57ca82ba47a04a92e7bafb055307447ec98f689426dd03d842e4fe1de9bd not found: ID does not exist"
Jul 17 06:42:36.164977 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.164951    2558 scope.go:117] "RemoveContainer" containerID="d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e"
Jul 17 06:42:36.165198 ip-10-0-140-96 kubenswrapper[2558]: E0717 06:42:36.165183    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e\": container with ID starting with d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e not found: ID does not exist" containerID="d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e"
Jul 17 06:42:36.165246 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.165201    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e"} err="failed to get container status \"d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e\": rpc error: code = NotFound desc = could not find container \"d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e\": container with ID starting with d4a633f1a3967a3e0b3020168e210a1e5a7cc4b8c86bb321fbadae32f0e47f3e not found: ID does not exist"
Jul 17 06:42:36.179778 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.179730    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"]
Jul 17 06:42:36.189371 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.189350    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-346c23-openshift-default-777bd9fb4c-hrxvv"]
Jul 17 06:42:36.634077 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.634029    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xzhjw_4dd38d67-dd44-4f64-af29-e876a1c83eb1/manager/0.log"
Jul 17 06:42:36.758851 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.758822    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ztlql_c74dbd36-746f-4a8f-bb08-6ab646a4836e/manager/0.log"
Jul 17 06:42:36.885779 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:36.885714    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64fd4654b6-gcwf9_54992e61-e79f-4679-92cd-f3028c00ff44/manager/0.log"
Jul 17 06:42:37.009538 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:37.009509    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-2tdtj_83a2d61b-6c06-4136-916c-01a82ff92723/server/0.log"
Jul 17 06:42:37.131249 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:37.131219    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jts89_a1ade4bd-5a9c-4a04-b311-ff623b15885d/manager/0.log"
Jul 17 06:42:37.511105 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:37.511076    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-554df894d5-zcxct_7a21be7f-68cb-4458-b0c6-9c2658c01da8/copy-manifests/0.log"
Jul 17 06:42:37.523431 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:37.523396    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-554df894d5-zcxct_7a21be7f-68cb-4458-b0c6-9c2658c01da8/manager/0.log"
Jul 17 06:42:37.916653 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:37.916620    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5bf885b94d-r9x75_a5f8b487-3631-4e74-995c-fff6d904423d/maas-api/0.log"
Jul 17 06:42:38.039711 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:38.039684    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737800-vqlhm_4a303bfb-b28a-4e61-a5cf-cf6e2742f10d/cleanup/0.log"
Jul 17 06:42:38.079256 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:38.079220    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" path="/var/lib/kubelet/pods/414677a6-4e6e-435b-8afc-c5696f6d8393/volumes"
Jul 17 06:42:38.151376 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:38.151289    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737815-bfzf4_98e314d1-895c-4b9f-a58b-e69257fe4851/cleanup/0.log"
Jul 17 06:42:40.989492 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:40.989444    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wpslm_4a3fadb5-1911-42c0-b52e-c3834d59cd41/manager/0.log"
Jul 17 06:42:41.104493 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:41.104449    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-47p8w_35abfa32-e9ef-4329-8e20-65b88a06c027/manager/0.log"
Jul 17 06:42:41.210809 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:41.210782    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-pxsl9_bde81c2f-1b0b-4adb-b100-abad3ec29772/kuadrant-console-plugin/0.log"
Jul 17 06:42:41.337588 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:41.337559    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rlnmc_9196e4da-33d8-4e46-acb8-411d0e81cd8c/registry-server/0.log"
Jul 17 06:42:41.479361 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:41.479334    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-j6tkg_c03a52bc-8bfe-4543-8bef-0d29efde9b16/manager/0.log"
Jul 17 06:42:41.721200 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:41.721120    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5sdjm_e571449b-4eee-42e9-8042-193c482d1bb0/manager/0.log"
Jul 17 06:42:42.075849 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:42.075734    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf42j8sn_a21ac819-78e6-4b7e-abbd-616e6bf6fb37/istio-proxy/0.log"
Jul 17 06:42:42.316721 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:42.316647    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79d55f4645-9qcsx_c53de3f8-1f8f-4688-b4c2-5c21a48e41e9/kube-auth-proxy/0.log"
Jul 17 06:42:42.557427 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:42.557397    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g2wg5_8cc064b1-bea1-41de-9e44-b10de3e4c1b3/istio-proxy/1.log"
Jul 17 06:42:43.506049 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:43.506015    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-qskbz_ab56ad30-c4d7-4b80-91ad-48d7bc5c9666/main/0.log"
Jul 17 06:42:43.628411 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:43.628380    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db25hfh_da6c99ee-9017-437f-a0eb-cac708960f9a/main/0.log"
Jul 17 06:42:49.062029 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.061996    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bbz7r/must-gather-4jmct"]
Jul 17 06:42:49.062411 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062399    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy"
Jul 17 06:42:49.062461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062413    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy"
Jul 17 06:42:49.062461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062421    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerName="revoke-keys"
Jul 17 06:42:49.062461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062427    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerName="revoke-keys"
Jul 17 06:42:49.062461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062444    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy"
Jul 17 06:42:49.062461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062449    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy"
Jul 17 06:42:49.062461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062457    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy"
Jul 17 06:42:49.062461 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062462    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062483    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f36db3c-2e3f-4af9-aeab-11d6bb91707a" containerName="main"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062489    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f36db3c-2e3f-4af9-aeab-11d6bb91707a" containerName="main"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062506    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerName="revoke-keys"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062512    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerName="revoke-keys"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062572    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062581    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerName="revoke-keys"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062589    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062595    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f36db3c-2e3f-4af9-aeab-11d6bb91707a" containerName="main"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062601    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerName="revoke-keys"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062608    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="414677a6-4e6e-435b-8afc-c5696f6d8393" containerName="istio-proxy"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062661    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062666    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062672    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerName="revoke-keys"
Jul 17 06:42:49.062710 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062677    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerName="revoke-keys"
Jul 17 06:42:49.063164 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062742    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="785df468-28af-4515-b0f0-5c97f5f588dc" containerName="istio-proxy"
Jul 17 06:42:49.063164 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.062751    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd17f6fd-d2fd-4d2c-b591-3b2e4d188764" containerName="revoke-keys"
Jul 17 06:42:49.067259 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.067242    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bbz7r/must-gather-4jmct"
Jul 17 06:42:49.070241 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.070213    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bbz7r\"/\"kube-root-ca.crt\""
Jul 17 06:42:49.070383 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.070243    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bbz7r\"/\"openshift-service-ca.crt\""
Jul 17 06:42:49.079162 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.079141    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bbz7r/must-gather-4jmct"]
Jul 17 06:42:49.132535 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.132514    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ac56390c-7f42-4d0e-b631-4789342c708e-must-gather-output\") pod \"must-gather-4jmct\" (UID: \"ac56390c-7f42-4d0e-b631-4789342c708e\") " pod="openshift-must-gather-bbz7r/must-gather-4jmct"
Jul 17 06:42:49.132649 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.132549    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhf4d\" (UniqueName: \"kubernetes.io/projected/ac56390c-7f42-4d0e-b631-4789342c708e-kube-api-access-bhf4d\") pod \"must-gather-4jmct\" (UID: \"ac56390c-7f42-4d0e-b631-4789342c708e\") " pod="openshift-must-gather-bbz7r/must-gather-4jmct"
Jul 17 06:42:49.233340 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.233316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ac56390c-7f42-4d0e-b631-4789342c708e-must-gather-output\") pod \"must-gather-4jmct\" (UID: \"ac56390c-7f42-4d0e-b631-4789342c708e\") " pod="openshift-must-gather-bbz7r/must-gather-4jmct"
Jul 17 06:42:49.233462 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.233349    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhf4d\" (UniqueName: \"kubernetes.io/projected/ac56390c-7f42-4d0e-b631-4789342c708e-kube-api-access-bhf4d\") pod \"must-gather-4jmct\" (UID: \"ac56390c-7f42-4d0e-b631-4789342c708e\") " pod="openshift-must-gather-bbz7r/must-gather-4jmct"
Jul 17 06:42:49.233653 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.233636    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ac56390c-7f42-4d0e-b631-4789342c708e-must-gather-output\") pod \"must-gather-4jmct\" (UID: \"ac56390c-7f42-4d0e-b631-4789342c708e\") " pod="openshift-must-gather-bbz7r/must-gather-4jmct"
Jul 17 06:42:49.247576 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.247559    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhf4d\" (UniqueName: \"kubernetes.io/projected/ac56390c-7f42-4d0e-b631-4789342c708e-kube-api-access-bhf4d\") pod \"must-gather-4jmct\" (UID: \"ac56390c-7f42-4d0e-b631-4789342c708e\") " pod="openshift-must-gather-bbz7r/must-gather-4jmct"
Jul 17 06:42:49.376375 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.376349    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bbz7r/must-gather-4jmct"
Jul 17 06:42:49.519421 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.519390    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bbz7r/must-gather-4jmct"]
Jul 17 06:42:49.520433 ip-10-0-140-96 kubenswrapper[2558]: W0717 06:42:49.520399    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac56390c_7f42_4d0e_b631_4789342c708e.slice/crio-e583e4daa35655789d229d5e91f79e3e359520170b5f02eacadd317004d4ad4e WatchSource:0}: Error finding container e583e4daa35655789d229d5e91f79e3e359520170b5f02eacadd317004d4ad4e: Status 404 returned error can't find the container with id e583e4daa35655789d229d5e91f79e3e359520170b5f02eacadd317004d4ad4e
Jul 17 06:42:49.522384 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:49.522349    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 06:42:50.232417 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:50.232380    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bbz7r/must-gather-4jmct" event={"ID":"ac56390c-7f42-4d0e-b631-4789342c708e","Type":"ContainerStarted","Data":"e583e4daa35655789d229d5e91f79e3e359520170b5f02eacadd317004d4ad4e"}
Jul 17 06:42:51.243222 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:51.242308    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bbz7r/must-gather-4jmct" event={"ID":"ac56390c-7f42-4d0e-b631-4789342c708e","Type":"ContainerStarted","Data":"81285e49caa294b44beedb7141759d6913d8d4e2ee178d74fbc6528b5ed1bb58"}
Jul 17 06:42:51.243222 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:51.242366    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bbz7r/must-gather-4jmct" event={"ID":"ac56390c-7f42-4d0e-b631-4789342c708e","Type":"ContainerStarted","Data":"0e51c1d3e17201b5ea59f8e9d77b6571530fb62752466edf3fccc8906ca388ea"}
Jul 17 06:42:51.263202 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:51.263140    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bbz7r/must-gather-4jmct" podStartSLOduration=1.493925998 podStartE2EDuration="2.263116998s" totalImagesPullingTime="769.191ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 06:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:42:51.261196541 +0000 UTC m=+3693.703135133" watchObservedRunningTime="2026-07-17 06:42:51.263116998 +0000 UTC m=+3693.705055573"
Jul 17 06:42:52.001986 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:52.001954    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h2lgh_1b659b41-1e15-4c23-95b6-0396eadae285/global-pull-secret-syncer/0.log"
Jul 17 06:42:52.153746 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:52.153718    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-crk4m_5a773d59-6bb2-4da5-97e6-53ae51a55cd7/konnectivity-agent/0.log"
Jul 17 06:42:52.261420 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:52.261335    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-96.ec2.internal_b714b8177e3d9d42f58f7241552dc9e9/haproxy/0.log"
Jul 17 06:42:57.168945 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:57.168869    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wpslm_4a3fadb5-1911-42c0-b52e-c3834d59cd41/manager/0.log"
Jul 17 06:42:57.196616 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:57.196578    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-47p8w_35abfa32-e9ef-4329-8e20-65b88a06c027/manager/0.log"
Jul 17 06:42:57.227857 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:57.227825    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-pxsl9_bde81c2f-1b0b-4adb-b100-abad3ec29772/kuadrant-console-plugin/0.log"
Jul 17 06:42:57.287405 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:57.287374    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rlnmc_9196e4da-33d8-4e46-acb8-411d0e81cd8c/registry-server/0.log"
Jul 17 06:42:57.488548 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:57.487465    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-j6tkg_c03a52bc-8bfe-4543-8bef-0d29efde9b16/manager/0.log"
Jul 17 06:42:57.610343 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:57.610288    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5sdjm_e571449b-4eee-42e9-8042-193c482d1bb0/manager/0.log"
Jul 17 06:42:59.479759 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:59.479707    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6c48c9cf85-78hkc_08563a71-8da4-4270-9370-e3ad8aaa4c9d/metrics-server/0.log"
Jul 17 06:42:59.505162 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:59.505131    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-5zxhh_76e15cb7-000d-4462-aceb-2cc1a38b23ab/monitoring-plugin/0.log"
Jul 17 06:42:59.534964 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:59.534857    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9wndm_97d15fd8-7e65-4d18-b11d-56854f7d380b/node-exporter/0.log"
Jul 17 06:42:59.558258 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:59.558222    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9wndm_97d15fd8-7e65-4d18-b11d-56854f7d380b/kube-rbac-proxy/0.log"
Jul 17 06:42:59.583695 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:42:59.583665    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9wndm_97d15fd8-7e65-4d18-b11d-56854f7d380b/init-textfile/0.log"
Jul 17 06:43:00.110088 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:00.110057    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-zjgh2_f878a236-a96b-4020-85fa-b5faa8813f00/prometheus-operator-admission-webhook/0.log"
Jul 17 06:43:00.228753 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:00.228727    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c959ff9bd-tm2sq_21c5ac76-68fa-4aaa-9125-bcf4944538f9/thanos-query/0.log"
Jul 17 06:43:00.257285 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:00.257260    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c959ff9bd-tm2sq_21c5ac76-68fa-4aaa-9125-bcf4944538f9/kube-rbac-proxy-web/0.log"
Jul 17 06:43:00.280181 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:00.280135    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c959ff9bd-tm2sq_21c5ac76-68fa-4aaa-9125-bcf4944538f9/kube-rbac-proxy/0.log"
Jul 17 06:43:00.305485 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:00.305440    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c959ff9bd-tm2sq_21c5ac76-68fa-4aaa-9125-bcf4944538f9/prom-label-proxy/0.log"
Jul 17 06:43:00.331180 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:00.331154    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c959ff9bd-tm2sq_21c5ac76-68fa-4aaa-9125-bcf4944538f9/kube-rbac-proxy-rules/0.log"
Jul 17 06:43:00.358622 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:00.358599    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c959ff9bd-tm2sq_21c5ac76-68fa-4aaa-9125-bcf4944538f9/kube-rbac-proxy-metrics/0.log"
Jul 17 06:43:01.552366 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:01.552335    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-hx4sr_ea296c95-407f-479a-819b-ecf699fb4f83/networking-console-plugin/0.log"
Jul 17 06:43:02.253274 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.253248    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/1.log"
Jul 17 06:43:02.260312 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.260285    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6qx5_2a485313-07ed-4e3e-a2b0-30770766d308/console-operator/2.log"
Jul 17 06:43:02.470975 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.470940    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"]
Jul 17 06:43:02.477962 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.477936    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.480794 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.480771    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bbz7r\"/\"default-dockercfg-rdkgz\""
Jul 17 06:43:02.489939 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.489914    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"]
Jul 17 06:43:02.584790 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.584753    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-sys\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.585171 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.584800    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-lib-modules\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.585171 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.584856    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgc7n\" (UniqueName: \"kubernetes.io/projected/563eb562-3194-48e2-9561-8e66183b1a12-kube-api-access-wgc7n\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.585171 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.584884    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-proc\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.585171 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.585011    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-podres\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.685962 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.685923    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgc7n\" (UniqueName: \"kubernetes.io/projected/563eb562-3194-48e2-9561-8e66183b1a12-kube-api-access-wgc7n\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.686161 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.685976    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-proc\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.686161 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.686023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-podres\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.686161 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.686074    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-sys\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.686161 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.686114    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-lib-modules\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.686378 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.686281    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-lib-modules\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.686378 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.686358    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-proc\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.686519 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.686413    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-sys\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.686519 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.686435    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/563eb562-3194-48e2-9561-8e66183b1a12-podres\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.695902 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.695864    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgc7n\" (UniqueName: \"kubernetes.io/projected/563eb562-3194-48e2-9561-8e66183b1a12-kube-api-access-wgc7n\") pod \"perf-node-gather-daemonset-n4ln9\" (UID: \"563eb562-3194-48e2-9561-8e66183b1a12\") " pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.791918 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.791877    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:02.970512 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:02.970466    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"]
Jul 17 06:43:03.398390 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:03.398352    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9" event={"ID":"563eb562-3194-48e2-9561-8e66183b1a12","Type":"ContainerStarted","Data":"754e20759c9a1b6b980d03d5773b3673006186efd11bbb8290b5c359d39883a6"}
Jul 17 06:43:03.398599 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:03.398415    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9" event={"ID":"563eb562-3194-48e2-9561-8e66183b1a12","Type":"ContainerStarted","Data":"4f9ab0f12bc61fbee01e17ff9aa3bffb2637dd685dd9812b18cda92463cae293"}
Jul 17 06:43:03.398599 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:03.398573    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:03.427944 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:03.427894    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9" podStartSLOduration=1.427878892 podStartE2EDuration="1.427878892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 06:43:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 06:43:03.424796204 +0000 UTC m=+3705.866734783" watchObservedRunningTime="2026-07-17 06:43:03.427878892 +0000 UTC m=+3705.869817484"
Jul 17 06:43:04.779809 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:04.779768    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9kcb2_ce41f40b-970f-4698-aa72-e7f772641eaf/dns/0.log"
Jul 17 06:43:04.811101 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:04.811052    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9kcb2_ce41f40b-970f-4698-aa72-e7f772641eaf/kube-rbac-proxy/0.log"
Jul 17 06:43:05.013909 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:05.013882    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lfzc2_2a31ea27-5883-4df9-9032-72ad9c0ee4ea/dns-node-resolver/0.log"
Jul 17 06:43:05.560274 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:05.560231    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-76dcbff9f5-lfp28_8887bdcd-8c10-401a-b2cb-bef7ac87108d/registry/0.log"
Jul 17 06:43:05.577863 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:05.577827    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-76dcbff9f5-lfp28_8887bdcd-8c10-401a-b2cb-bef7ac87108d/registry/1.log"
Jul 17 06:43:05.609417 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:05.609389    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6drsh_19e2915f-ebfc-45ff-9452-9e0c35a105ca/node-ca/0.log"
Jul 17 06:43:06.738504 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:06.738457    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf42j8sn_a21ac819-78e6-4b7e-abbd-616e6bf6fb37/istio-proxy/0.log"
Jul 17 06:43:06.928052 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:06.928020    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79d55f4645-9qcsx_c53de3f8-1f8f-4688-b4c2-5c21a48e41e9/kube-auth-proxy/0.log"
Jul 17 06:43:07.039336 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:07.039257    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g2wg5_8cc064b1-bea1-41de-9e44-b10de3e4c1b3/istio-proxy/0.log"
Jul 17 06:43:07.074194 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:07.074156    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g2wg5_8cc064b1-bea1-41de-9e44-b10de3e4c1b3/istio-proxy/1.log"
Jul 17 06:43:08.099487 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:08.099451    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dwxmp_480c28d5-a4e5-4bef-9605-a0ceff3cc4b6/serve-healthcheck-canary/0.log"
Jul 17 06:43:08.824807 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:08.824784    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2ktjt_2ad0ac4a-8e85-4e71-9970-cd21695740fb/kube-rbac-proxy/0.log"
Jul 17 06:43:08.861836 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:08.861806    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2ktjt_2ad0ac4a-8e85-4e71-9970-cd21695740fb/exporter/0.log"
Jul 17 06:43:08.906944 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:08.906841    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2ktjt_2ad0ac4a-8e85-4e71-9970-cd21695740fb/extractor/0.log"
Jul 17 06:43:09.416256 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:09.416228    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bbz7r/perf-node-gather-daemonset-n4ln9"
Jul 17 06:43:11.994834 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:11.994755    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xzhjw_4dd38d67-dd44-4f64-af29-e876a1c83eb1/manager/0.log"
Jul 17 06:43:12.254398 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:12.254316    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ztlql_c74dbd36-746f-4a8f-bb08-6ab646a4836e/manager/0.log"
Jul 17 06:43:12.412912 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:12.412877    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64fd4654b6-gcwf9_54992e61-e79f-4679-92cd-f3028c00ff44/manager/0.log"
Jul 17 06:43:12.465537 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:12.465496    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf42j8sn_a21ac819-78e6-4b7e-abbd-616e6bf6fb37/istio-proxy/0.log"
Jul 17 06:43:12.477394 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:12.477373    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-2tdtj_83a2d61b-6c06-4136-916c-01a82ff92723/server/0.log"
Jul 17 06:43:12.633868 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:12.633831    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jts89_a1ade4bd-5a9c-4a04-b311-ff623b15885d/manager/0.log"
Jul 17 06:43:12.661743 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:12.661713    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79d55f4645-9qcsx_c53de3f8-1f8f-4688-b4c2-5c21a48e41e9/kube-auth-proxy/0.log"
Jul 17 06:43:12.741697 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:12.741668    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g2wg5_8cc064b1-bea1-41de-9e44-b10de3e4c1b3/istio-proxy/0.log"
Jul 17 06:43:12.789328 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:12.789294    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g2wg5_8cc064b1-bea1-41de-9e44-b10de3e4c1b3/istio-proxy/1.log"
Jul 17 06:43:13.130049 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:13.130024    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-554df894d5-zcxct_7a21be7f-68cb-4458-b0c6-9c2658c01da8/manager/0.log"
Jul 17 06:43:13.203352 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:13.203328    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-554df894d5-zcxct_7a21be7f-68cb-4458-b0c6-9c2658c01da8/copy-manifests/0.log"
Jul 17 06:43:14.335450 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:14.335417    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf42j8sn_a21ac819-78e6-4b7e-abbd-616e6bf6fb37/istio-proxy/0.log"
Jul 17 06:43:14.479465 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:14.479435    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79d55f4645-9qcsx_c53de3f8-1f8f-4688-b4c2-5c21a48e41e9/kube-auth-proxy/0.log"
Jul 17 06:43:14.529230 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:14.529177    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g2wg5_8cc064b1-bea1-41de-9e44-b10de3e4c1b3/istio-proxy/0.log"
Jul 17 06:43:14.574795 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:14.574719    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g2wg5_8cc064b1-bea1-41de-9e44-b10de3e4c1b3/istio-proxy/1.log"
Jul 17 06:43:19.961872 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:19.961834    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-rbl5t_5d81fdd7-e4aa-4faf-a69e-588eac54439c/migrator/0.log"
Jul 17 06:43:19.988621 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:19.988598    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-rbl5t_5d81fdd7-e4aa-4faf-a69e-588eac54439c/graceful-termination/0.log"
Jul 17 06:43:20.442178 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:20.442145    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-dnprn_1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7/kube-storage-version-migrator-operator/1.log"
Jul 17 06:43:20.443289 ip-10-0-140-96 kubenswrapper[2558]: I0717 06:43:20.443265    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-dnprn_1d95d07d-b0e8-4adb-86ef-4ea16b1a21c7/kube-storage-version-migrator-operator/0.log"