Jul 07 23:13:33.635733 ip-10-0-137-77 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 23:13:35.679049 ip-10-0-137-77 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 23:13:35.679049 ip-10-0-137-77 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 23:13:35.679049 ip-10-0-137-77 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 23:13:35.679049 ip-10-0-137-77 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 23:13:35.679049 ip-10-0-137-77 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 23:13:35.679789 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.679701    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 23:13:35.683991 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.683973    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 23:13:35.683991 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.683991    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.683994    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.683998    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684001    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684004    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684007    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684010    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684013    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684017    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684020    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684022    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684025    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684027    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684030    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684033    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684035    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684038    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684041    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684044    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684047    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684049    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 23:13:35.684055 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684052    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684054    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684057    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684060    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684063    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684066    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684068    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684072    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684075    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684079    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684083    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684085    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684088    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684092    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684094    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684096    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684099    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684101    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684105    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684107    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 23:13:35.684581 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684110    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684112    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684115    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684117    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684120    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684122    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684125    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684127    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684130    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684133    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684135    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684138    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684140    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684142    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684146    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684149    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684151    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684155    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684157    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684160    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 23:13:35.685083 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684163    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684165    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684168    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684170    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684174    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684177    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684180    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684182    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684185    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684188    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684190    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684193    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684195    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684198    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684201    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684203    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684206    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684209    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684213    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 23:13:35.685584 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684216    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684219    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684222    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684225    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684228    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684230    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684233    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684237    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684240    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684244    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684247    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684251    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684253    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684256    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684259    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684262    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684264    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.684267    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685020    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 23:13:35.686040 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685027    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685030    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685032    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685035    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685038    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685041    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685045    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685047    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685050    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685052    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685055    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685057    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685060    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685063    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685066    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685068    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685071    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685074    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685077    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685079    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 23:13:35.686517 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685082    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685085    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685088    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685091    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685094    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685097    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685099    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685102    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685104    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685107    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685110    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685112    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685114    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685117    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685121    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685125    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685129    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685133    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685136    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 23:13:35.687042 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685140    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685142    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685145    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685148    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685151    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685153    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685156    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685158    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685161    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685164    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685167    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685170    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685172    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685175    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685177    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685180    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685183    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685185    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685188    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685191    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 23:13:35.687496 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685195    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685197    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685200    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685203    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685205    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685208    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685210    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685213    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685215    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685218    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685220    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685224    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685227    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685230    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685233    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685235    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685238    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685240    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685246    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 23:13:35.687975 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685249    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685252    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685255    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685258    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685261    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685263    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685266    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685269    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685272    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685274    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685277    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685280    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685283    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685285    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685287    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685299    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685304    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685307    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685310    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.685313    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 23:13:35.688414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686342    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686349    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686352    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686356    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686366    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686369    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686372    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686375    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686378    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686381    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686384    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686386    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686389    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686391    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686394    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686396    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686398    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686401    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686404    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686406    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 23:13:35.688911 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686409    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686411    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686414    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686416    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686419    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686421    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686425    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686429    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686433    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686435    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686438    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686441    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686444    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686447    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686449    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686452    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686455    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686463    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686466    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 23:13:35.689403 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686469    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686471    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686473    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686476    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686478    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686481    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686483    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686485    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686488    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686490    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686493    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686495    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686498    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686500    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686503    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686505    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686508    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686510    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686513    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 23:13:35.689881 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686516    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686519    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686522    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686525    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686529    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686532    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686535    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686538    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686541    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686544    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686549    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686552    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686572    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686575    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686577    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686580    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686582    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686585    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686587    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686590    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 23:13:35.690336 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686592    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686595    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686597    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686600    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686602    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686605    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686608    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686610    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686612    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686615    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686617    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686619    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686622    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686624    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686627    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686631    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686635    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686638    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686640    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 23:13:35.690824 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686643    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.686646    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686719    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686727    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686734    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686738    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686750    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686754    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686758    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686763    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686767    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686770    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686774    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686777    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686780    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686783    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686786    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686789    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686792    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686794    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686799    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686802    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686805    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686808    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 23:13:35.691279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686812    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686816    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686819    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686822    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686825    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686829    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686831    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686834    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686838    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686840    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686844    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686849    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686852    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686855    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686857    2578 flags.go:64] FLAG: --enable-server="true"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686861    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686865    2578 flags.go:64] FLAG: --event-burst="100"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686868    2578 flags.go:64] FLAG: --event-qps="50"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686871    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686874    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686877    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686881    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686884    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686887    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686890    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 07 23:13:35.691865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686893    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686896    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686899    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686902    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686905    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686907    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686910    2578 flags.go:64] FLAG: --feature-gates=""
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686914    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686917    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686920    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686925    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686928    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686931    2578 flags.go:64] FLAG: --help="false"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686934    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-137-77.ec2.internal"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686937    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686940    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686942    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686946    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686950    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686953    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686956    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686959    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686962    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 23:13:35.692484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686965    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686968    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686971    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686974    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686977    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686980    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686983    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686986    2578 flags.go:64] FLAG: --lock-file=""
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686988    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686991    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.686994    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687000    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687003    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687006    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687008    2578 flags.go:64] FLAG: --logging-format="text"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687011    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687014    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687017    2578 flags.go:64] FLAG: --manifest-url=""
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687020    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687026    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687029    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687032    2578 flags.go:64] FLAG: --max-pods="110"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687035    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687039    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687041    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 23:13:35.693060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687044    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687048    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687052    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687055    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687062    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687065    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687068    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687071    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687075    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687080    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687083    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687087    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687090    2578 flags.go:64] FLAG: --port="10250"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687092    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687095    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b464741d08245fff"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687098    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687101    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687104    2578 flags.go:64] FLAG: --register-node="true"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687108    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687112    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687115    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687118    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687120    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687124    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687127    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 23:13:35.693723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687130    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687134    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687137    2578 flags.go:64] FLAG: --runonce="false"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687140    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687143    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687146    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687149    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687152    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687155    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687158    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687162    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687165    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687168    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687171    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687174    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687177    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687180    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687183    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687188    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687191    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687194    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687199    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687202    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687204    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687207    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 23:13:35.694649 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687211    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687213    2578 flags.go:64] FLAG: --v="2"
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687217    2578 flags.go:64] FLAG: --version="false"
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687221    2578 flags.go:64] FLAG: --vmodule=""
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687225    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.687228    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687386    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687391    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687396    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687399    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687403    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687407    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687410    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687413    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687416    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687418    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687421    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687426    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687428    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687431    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687434    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 23:13:35.695687 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687436    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687439    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687441    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687444    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687447    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687450    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687452    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687455    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687457    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687460    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687463    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687466    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687468    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687470    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687473    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687475    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687478    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687480    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687483    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 23:13:35.696548 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687487    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687489    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687492    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687494    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687497    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687499    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687502    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687505    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687508    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687511    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687514    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687516    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687519    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687521    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687524    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687526    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687529    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687532    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687534    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687537    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 23:13:35.697444 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687541    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687544    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687547    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687550    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687552    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687555    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687570    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687574    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687577    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687580    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687582    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687585    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687588    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687591    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687594    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687596    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687599    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687602    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687604    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687607    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 23:13:35.698204 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687609    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687612    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687615    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687618    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687620    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687624    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687628    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687631    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687633    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687635    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687639    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687641    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687643    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687646    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687648    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687651    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687653    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687656    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687658    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 23:13:35.698870 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687661    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687664    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687666    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687669    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687671    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.687675    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.688444    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.696236    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.696260    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.696285    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.696294    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696380    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696388    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696394    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696399    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 23:13:35.699607 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696404    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696408    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696413    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696418    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696422    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696427    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696432    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696437    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696441    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696446    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696450    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696455    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696459    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696464    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696469    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696475    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696481    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696486    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696491    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 23:13:35.700224 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696495    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696499    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696504    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696508    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696512    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696517    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696524    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696528    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696533    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696537    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696541    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696545    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696550    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696573    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696580    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696584    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696589    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696593    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696598    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696602    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 23:13:35.700903 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696607    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696612    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696616    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696621    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696625    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696630    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696634    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696639    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696644    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696648    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696653    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696657    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696661    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696665    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696669    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696674    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696678    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696682    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696687    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696692    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696698    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 23:13:35.701391 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696702    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696707    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696711    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696715    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696719    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696723    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696727    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696732    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696737    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696741    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696745    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696750    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696754    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696758    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696762    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696767    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696771    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696775    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696779    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 23:13:35.702016 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696784    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696788    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696793    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696798    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696802    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696806    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696813    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696819    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696824    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696828    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696833    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696837    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696842    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696847    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696851    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 23:13:35.702482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.696857    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.696865    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697048    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697059    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697066    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697070    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697075    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697079    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697083    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697087    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697116    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697123    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697128    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697133    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 23:13:35.702962 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697138    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697143    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697148    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697153    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697157    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697162    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697166    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697170    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697175    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697179    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697183    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697187    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697192    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697196    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697200    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697204    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697209    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697213    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697218    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697223    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 23:13:35.703300 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697227    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697231    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697235    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697239    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697243    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697247    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697251    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697255    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697259    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697263    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697270    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697275    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697280    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697283    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697287    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697299    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697304    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697309    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697314    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697319    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 23:13:35.703799 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697323    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697328    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697332    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697336    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697341    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697345    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697349    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697354    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697358    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697369    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697374    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697378    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697382    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697387    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697391    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697395    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697399    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697403    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697407    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697411    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 23:13:35.704268 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697416    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697420    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697424    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697429    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697433    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697437    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697440    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697444    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697452    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697455    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697459    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697463    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697467    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697471    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697475    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697479    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697483    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697488    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697492    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 23:13:35.704765 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697496    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697503    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697509    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697514    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697519    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697524    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697528    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:35.697533    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.697542    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.697736    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.702169    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.703703    2578 server.go:1019] "Starting client certificate rotation"
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.703801    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 23:13:35.705233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.703832    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 23:13:35.706207 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.706192    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 23:13:35.707341 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.707325    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 23:13:35.714659 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.714641    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 07 23:13:35.717795 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.717768    2578 log.go:25] "Validated CRI v1 image API"
Jul 07 23:13:35.718604 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.718587    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 23:13:35.721502 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.721463    2578 fs.go:135] Filesystem UUIDs: map[0e1be87c-abd8-46b1-8352-0780abbfb301:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e94ba1a8-57d3-447a-9f71-c461ebaa4b88:/dev/nvme0n1p3]
Jul 07 23:13:35.721601 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.721499    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 23:13:35.727091 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.726962    2578 manager.go:216] Machine: {Timestamp:2026-07-07 23:13:35.726463685 +0000 UTC m=+0.312033042 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099940 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d7670c8dc3a863552100a18e9755f SystemUUID:ec2d7670-c8dc-3a86-3552-100a18e9755f BootID:db13a742-54d2-44e4-92d5-97dd3e507093 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cc:68:5b:b6:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cc:68:5b:b6:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:4c:f6:6b:82:b5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 23:13:35.727091 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.727081    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 23:13:35.727213 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.727164    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 23:13:35.730173 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.730143    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 23:13:35.730347 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.730178    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-77.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 23:13:35.731013 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.731000    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 23:13:35.731013 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.731015    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 23:13:35.731090 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.731023    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 23:13:35.731537 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.731526    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 23:13:35.732897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.732882    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 23:13:35.733927 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.733901    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 23:13:35.736302 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.736284    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 23:13:35.736460 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.736449    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 23:13:35.736518 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.736488    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 23:13:35.736518 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.736508    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 23:13:35.736630 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.736530    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 23:13:35.736630 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.736548    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 07 23:13:35.736630 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.736577    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 23:13:35.737052 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.737036    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 23:13:35.737112 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.737067    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 23:13:35.741012 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.740929    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 23:13:35.743319 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.743299    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 23:13:35.743398 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.743332    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 23:13:35.744281 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744269    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 23:13:35.744334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744285    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 23:13:35.744334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744293    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 23:13:35.744334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744298    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 23:13:35.744334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744307    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 23:13:35.744334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744313    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 23:13:35.744334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744320    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 23:13:35.744334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744326    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 23:13:35.744334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744333    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 23:13:35.744539 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744339    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 23:13:35.744539 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744347    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 23:13:35.744539 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744352    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 23:13:35.744539 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744375    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 23:13:35.744539 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.744381    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 23:13:35.747229 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.747207    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-77.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 23:13:35.747308 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.747245    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 23:13:35.747308 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.747289    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 23:13:35.748003 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.747990    2578 server.go:1268] "Started kubelet"
Jul 07 23:13:35.748111 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.748083    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 23:13:35.748159 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.748083    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 23:13:35.748203 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.748186    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 23:13:35.749061 ip-10-0-137-77 systemd[1]: Started Kubernetes Kubelet.
Jul 07 23:13:35.750226 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.750167    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 23:13:35.751115 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.751096    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 23:13:35.752098 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.752070    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 23:13:35.755409 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.754519    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-77.ec2.internal.18c024e461a37071  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-77.ec2.internal,UID:ip-10-0-137-77.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-77.ec2.internal,},FirstTimestamp:2026-07-07 23:13:35.747960945 +0000 UTC m=+0.333530297,LastTimestamp:2026-07-07 23:13:35.747960945 +0000 UTC m=+0.333530297,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-77.ec2.internal,}"
Jul 07 23:13:35.757885 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.757866    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 23:13:35.757970 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.757871    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 23:13:35.758059 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.758031    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:35.758784 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.758485    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 23:13:35.758784 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.758505    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 23:13:35.758784 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.758483    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 23:13:35.758784 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.758671    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 23:13:35.758784 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.758679    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 23:13:35.759121 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.758823    2578 factory.go:55] Registering systemd factory
Jul 07 23:13:35.759121 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.758842    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 07 23:13:35.759121 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.759046    2578 factory.go:153] Registering CRI-O factory
Jul 07 23:13:35.759121 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.759057    2578 factory.go:223] Registration of the crio container factory successfully
Jul 07 23:13:35.759121 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.759095    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 23:13:35.759283 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.759123    2578 factory.go:103] Registering Raw factory
Jul 07 23:13:35.759283 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.759149    2578 manager.go:1196] Started watching for new ooms in manager
Jul 07 23:13:35.759713 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.759701    2578 manager.go:349] Starting recovery of all containers
Jul 07 23:13:35.760724 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.760701    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lc2rz"
Jul 07 23:13:35.761001 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.760965    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 23:13:35.761123 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.761103    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 23:13:35.769115 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.769085    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lc2rz"
Jul 07 23:13:35.772576 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.772399    2578 manager.go:354] Recovery completed
Jul 07 23:13:35.777805 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.777789    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 23:13:35.779704 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.779687    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 23:13:35.779769 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.779718    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 23:13:35.779769 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.779729    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientPID"
Jul 07 23:13:35.780276 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.780264    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 23:13:35.780276 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.780277    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 23:13:35.780392 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.780296    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 23:13:35.780392 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.780276    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 23:13:35.781064 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.781050    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 23:13:35.781127 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.781078    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 23:13:35.781127 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.781091    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientPID"
Jul 07 23:13:35.781682 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.781670    2578 policy_none.go:49] "None policy: Start"
Jul 07 23:13:35.781718 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.781689    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 23:13:35.781718 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.781701    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 23:13:35.781896 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.781821    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-77.ec2.internal.18c024e46387cb8e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-77.ec2.internal,UID:ip-10-0-137-77.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-77.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-77.ec2.internal,},FirstTimestamp:2026-07-07 23:13:35.779703694 +0000 UTC m=+0.365273045,LastTimestamp:2026-07-07 23:13:35.779703694 +0000 UTC m=+0.365273045,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-77.ec2.internal,}"
Jul 07 23:13:35.782617 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.782605    2578 policy_none.go:47] "Start"
Jul 07 23:13:35.821226 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.821199    2578 manager.go:337] "Starting Device Plugin manager"
Jul 07 23:13:35.821378 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.821262    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 23:13:35.821378 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.821281    2578 server.go:87] "Starting device plugin registration server"
Jul 07 23:13:35.821601 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.821589    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 23:13:35.821663 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.821602    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 23:13:35.821853 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.821835    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 23:13:35.822010 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.821967    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 23:13:35.822010 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.821977    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 23:13:35.822444 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.822421    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 23:13:35.822528 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.822473    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:35.921768 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.921737    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 23:13:35.924149 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.922690    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 23:13:35.924149 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.922718    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 23:13:35.924149 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.922728    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientPID"
Jul 07 23:13:35.924149 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.922755    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-77.ec2.internal"
Jul 07 23:13:35.932029 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.932009    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-77.ec2.internal"
Jul 07 23:13:35.932094 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.932036    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-77.ec2.internal\": node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:35.950489 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.950462    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:35.954795 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.954760    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 23:13:35.956048 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.956028    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 23:13:35.956134 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.956080    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 23:13:35.956134 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.956120    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 23:13:35.956206 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:35.956164    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 23:13:35.958575 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:35.958542    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 23:13:36.050636 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.050610    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:36.056821 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.056758    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-77.ec2.internal"]
Jul 07 23:13:36.056891 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.056842    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 23:13:36.057845 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.057830    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 23:13:36.057915 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.057861    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 23:13:36.057915 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.057871    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientPID"
Jul 07 23:13:36.060066 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.060047    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d0b20c97249348f78521eef655be2a93-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal\" (UID: \"d0b20c97249348f78521eef655be2a93\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.060111 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.060075    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0b20c97249348f78521eef655be2a93-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal\" (UID: \"d0b20c97249348f78521eef655be2a93\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.060399 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.060386    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 23:13:36.060532 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.060518    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.060583 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.060550    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 23:13:36.061285 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.061269    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 23:13:36.061371 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.061306    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 23:13:36.061371 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.061308    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 23:13:36.061371 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.061336    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 23:13:36.061371 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.061350    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientPID"
Jul 07 23:13:36.061371 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.061317    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientPID"
Jul 07 23:13:36.063784 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.063769    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.063851 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.063795    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 23:13:36.064583 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.064548    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 23:13:36.064661 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.064594    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 23:13:36.064661 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.064604    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeHasSufficientPID"
Jul 07 23:13:36.090409 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.090382    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-77.ec2.internal\" not found" node="ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.094838 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.094816    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-77.ec2.internal\" not found" node="ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.151597 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.151540    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:36.160908 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.160884    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d0b20c97249348f78521eef655be2a93-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal\" (UID: \"d0b20c97249348f78521eef655be2a93\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.160970 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.160915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0b20c97249348f78521eef655be2a93-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal\" (UID: \"d0b20c97249348f78521eef655be2a93\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.160970 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.160940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f393c9727e900c86fef1db62fb4e666a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-77.ec2.internal\" (UID: \"f393c9727e900c86fef1db62fb4e666a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.161076 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.160995    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d0b20c97249348f78521eef655be2a93-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal\" (UID: \"d0b20c97249348f78521eef655be2a93\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.161076 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.161061    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0b20c97249348f78521eef655be2a93-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal\" (UID: \"d0b20c97249348f78521eef655be2a93\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.252298 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.252251    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:36.261709 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.261673    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f393c9727e900c86fef1db62fb4e666a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-77.ec2.internal\" (UID: \"f393c9727e900c86fef1db62fb4e666a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.261770 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.261733    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f393c9727e900c86fef1db62fb4e666a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-77.ec2.internal\" (UID: \"f393c9727e900c86fef1db62fb4e666a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.352925 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.352831    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:36.394326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.394298    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.399368 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.399259    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:36.453380 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.453342    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:36.553936 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.553914    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:36.609761 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.609677    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 23:13:36.654905 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.654871    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:36.703363 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.703321    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 23:13:36.703898 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.703499    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 23:13:36.703898 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.703515    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 23:13:36.755053 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.755024    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:36.771672 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.771627    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 23:08:35 +0000 UTC" deadline="2028-03-01 02:51:41.88201483 +0000 UTC"
Jul 07 23:13:36.771672 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.771662    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14451h38m5.110355505s"
Jul 07 23:13:36.856130 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.856102    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:36.956700 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:36.956627    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-77.ec2.internal\" not found"
Jul 07 23:13:36.977684 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:36.977647    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0b20c97249348f78521eef655be2a93.slice/crio-ca40ba1e44d2280430a765e69d8fae6f5013e6a6fa57026ee24cfc3d5ea1ef92 WatchSource:0}: Error finding container ca40ba1e44d2280430a765e69d8fae6f5013e6a6fa57026ee24cfc3d5ea1ef92: Status 404 returned error can't find the container with id ca40ba1e44d2280430a765e69d8fae6f5013e6a6fa57026ee24cfc3d5ea1ef92
Jul 07 23:13:36.977905 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:36.977884    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf393c9727e900c86fef1db62fb4e666a.slice/crio-cb0e8f96c96dc998b2358449051d77e55a6df2dd4de1f08763b27db7f16d7f94 WatchSource:0}: Error finding container cb0e8f96c96dc998b2358449051d77e55a6df2dd4de1f08763b27db7f16d7f94: Status 404 returned error can't find the container with id cb0e8f96c96dc998b2358449051d77e55a6df2dd4de1f08763b27db7f16d7f94
Jul 07 23:13:36.983414 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:36.983398    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 23:13:37.027724 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.027696    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 23:13:37.058998 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.058978    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:37.072963 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.072943    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 23:13:37.073946 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.073932    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-77.ec2.internal"
Jul 07 23:13:37.083133 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.083106    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 23:13:37.333028 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.332952    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 23:13:37.693758 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.693665    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 23:13:37.738402 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.738371    2578 apiserver.go:52] "Watching apiserver"
Jul 07 23:13:37.751548 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.751515    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 23:13:37.753919 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.753872    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-wv855","openshift-cluster-node-tuning-operator/tuned-6mqgz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal","openshift-network-diagnostics/network-check-target-v6x6f","openshift-network-operator/iptables-alerter-tlptd","openshift-ovn-kubernetes/ovnkube-node-57lng","kube-system/global-pull-secret-syncer-5kxrk","kube-system/kube-apiserver-proxy-ip-10-0-137-77.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42","openshift-dns/node-resolver-7kqxl","openshift-image-registry/node-ca-g7tzd","openshift-multus/multus-additional-cni-plugins-5lxn5","openshift-multus/multus-fjbhl","openshift-multus/network-metrics-daemon-mr5rb"]
Jul 07 23:13:37.756822 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.756796    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tlptd"
Jul 07 23:13:37.758964 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.758051    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 23:13:37.760020 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.759995    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:37.760133 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:37.760071    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:37.762933 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.762910    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-78k5x\""
Jul 07 23:13:37.763759 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.763549    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 23:13:37.763759 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.763551    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 23:13:37.763972 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.763956    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.766651 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.766628    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:37.766750 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:37.766698    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:13:37.767905 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.767883    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 23:13:37.768595 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.768423    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 23:13:37.769692 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.768958    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:37.769692 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:37.769021    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:37.769692 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769387    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 23:13:37.769692 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769428    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 23:13:37.769692 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769480    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lbzf7\""
Jul 07 23:13:37.769692 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/337b8d66-22e8-4cee-a345-c5dd54dca37b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769724    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e8e98a3-610a-463f-8546-0af03fe1f4f3-iptables-alerter-script\") pod \"iptables-alerter-tlptd\" (UID: \"1e8e98a3-610a-463f-8546-0af03fe1f4f3\") " pod="openshift-network-operator/iptables-alerter-tlptd"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769748    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2pxb\" (UniqueName: \"kubernetes.io/projected/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-kube-api-access-q2pxb\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769797    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-cnibin\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769822    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/337b8d66-22e8-4cee-a345-c5dd54dca37b-cni-binary-copy\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e8e98a3-610a-463f-8546-0af03fe1f4f3-host-slash\") pod \"iptables-alerter-tlptd\" (UID: \"1e8e98a3-610a-463f-8546-0af03fe1f4f3\") " pod="openshift-network-operator/iptables-alerter-tlptd"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769889    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769887    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-system-cni-dir\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769949    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/337b8d66-22e8-4cee-a345-c5dd54dca37b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.769987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzctr\" (UniqueName: \"kubernetes.io/projected/1e8e98a3-610a-463f-8546-0af03fe1f4f3-kube-api-access-hzctr\") pod \"iptables-alerter-tlptd\" (UID: \"1e8e98a3-610a-463f-8546-0af03fe1f4f3\") " pod="openshift-network-operator/iptables-alerter-tlptd"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.770016    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-os-release\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.770152 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.770040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz772\" (UniqueName: \"kubernetes.io/projected/337b8d66-22e8-4cee-a345-c5dd54dca37b-kube-api-access-pz772\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.771347 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.771329    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.771848 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.771602    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 23:13:37.774428 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.774412    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:13:37.774652 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.774635    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 23:13:37.776733 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.776369    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.777913 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.777893    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fkh87\""
Jul 07 23:13:37.778716 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.778700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7kqxl"
Jul 07 23:13:37.781331 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.780472    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 23:13:37.781331 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.780610    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 23:13:37.781331 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.781100    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 23:13:37.781331 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.781110    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 23:13:37.781514 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.781337    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g7tzd"
Jul 07 23:13:37.781807 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.781788    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 23:13:37.782451 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.782430    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 23:13:37.782714 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.782696    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jcsgn\""
Jul 07 23:13:37.782943 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.782909    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 23:13:37.783222 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.783206    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 23:13:37.783498 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.783481    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 23:13:37.783745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.783727    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 23:13:37.783962 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.783946    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r6xxm\""
Jul 07 23:13:37.784209 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.784188    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 23:13:37.784281 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.784250    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 23:13:37.784335 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.784201    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vm876\""
Jul 07 23:13:37.784470 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.784436    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 23:13:37.784574 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.784488    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lw7gj\""
Jul 07 23:13:37.784574 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.784491    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 23:13:37.784724 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.784491    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 23:13:37.784724 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.784693    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 23:13:37.785342 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.785323    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.788013 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.787991    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.789275 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.788710    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 23:13:37.789275 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.788886    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qhzgl\""
Jul 07 23:13:37.790862 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.790803    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 23:13:37.791041 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.791023    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hp7s5\""
Jul 07 23:13:37.791406 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.791387    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 23:13:37.797320 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.797286    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2h59v"
Jul 07 23:13:37.808491 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.808465    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2h59v"
Jul 07 23:13:37.859924 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.859894    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 23:13:37.870507 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870473    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-lib-modules\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.870691 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870523    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz772\" (UniqueName: \"kubernetes.io/projected/337b8d66-22e8-4cee-a345-c5dd54dca37b-kube-api-access-pz772\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.870691 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqmcz\" (UniqueName: \"kubernetes.io/projected/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-kube-api-access-qqmcz\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.870691 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870585    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-kubelet-config\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:37.870691 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870607    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:37.870691 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-socket-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.870691 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870654    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-etc-selinux\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.870691 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870676    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z84lc\" (UniqueName: \"kubernetes.io/projected/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-kube-api-access-z84lc\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.871049 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870720    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c342d1b2-9987-404d-88ae-197de63d7d04-serviceca\") pod \"node-ca-g7tzd\" (UID: \"c342d1b2-9987-404d-88ae-197de63d7d04\") " pod="openshift-image-registry/node-ca-g7tzd"
Jul 07 23:13:37.871049 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-run-openvswitch\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.871049 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/af219d19-df59-4d6d-97a2-8985f717797c-env-overrides\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.871049 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-var-lib-cni-bin\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.871049 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.870898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-tuned\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.871049 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871010    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-cnibin\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.871049 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871038    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjvqc\" (UniqueName: \"kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc\") pod \"network-check-target-v6x6f\" (UID: \"5785bd92-cfb4-4235-9d78-47c6884049c6\") " pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:37.871372 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871082    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc82q\" (UniqueName: \"kubernetes.io/projected/99cf80ee-8d7c-47a0-b546-13487e27e5cf-kube-api-access-pc82q\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.871372 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871094    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-cnibin\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.871372 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871106    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-modprobe-d\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.871372 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e8e98a3-610a-463f-8546-0af03fe1f4f3-host-slash\") pod \"iptables-alerter-tlptd\" (UID: \"1e8e98a3-610a-463f-8546-0af03fe1f4f3\") " pod="openshift-network-operator/iptables-alerter-tlptd"
Jul 07 23:13:37.871372 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871266    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/af219d19-df59-4d6d-97a2-8985f717797c-ovnkube-script-lib\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.871372 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871332    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e8e98a3-610a-463f-8546-0af03fe1f4f3-host-slash\") pod \"iptables-alerter-tlptd\" (UID: \"1e8e98a3-610a-463f-8546-0af03fe1f4f3\") " pod="openshift-network-operator/iptables-alerter-tlptd"
Jul 07 23:13:37.871662 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-cni-dir\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.871662 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871485    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-cnibin\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.871662 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871606    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-hostroot\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.871662 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-run\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.871848 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-var-lib-kubelet\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.871848 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871793    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-log-socket\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.871848 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871821    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-os-release\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.871986 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871855    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-run-ovn\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.871986 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871878    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-run-netns\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.871986 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871963    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-os-release\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.871986 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.871964    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-run-multus-certs\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.872151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872030    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-tmp\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.872151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9xjj\" (UniqueName: \"kubernetes.io/projected/f1a6d6ed-76dd-45b7-bfc7-65f0babd3893-kube-api-access-x9xjj\") pod \"node-resolver-7kqxl\" (UID: \"f1a6d6ed-76dd-45b7-bfc7-65f0babd3893\") " pod="openshift-dns/node-resolver-7kqxl"
Jul 07 23:13:37.872151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-cni-bin\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.872151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872101    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-os-release\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.872151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872123    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-socket-dir-parent\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.872151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-var-lib-cni-multus\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.872443 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872164    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1a6d6ed-76dd-45b7-bfc7-65f0babd3893-tmp-dir\") pod \"node-resolver-7kqxl\" (UID: \"f1a6d6ed-76dd-45b7-bfc7-65f0babd3893\") " pod="openshift-dns/node-resolver-7kqxl"
Jul 07 23:13:37.872443 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/337b8d66-22e8-4cee-a345-c5dd54dca37b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.872443 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872210    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-slash\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.872443 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872231    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-etc-openvswitch\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.872443 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-sysctl-conf\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.872443 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872274    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-host\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.872443 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872383    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f1a6d6ed-76dd-45b7-bfc7-65f0babd3893-hosts-file\") pod \"node-resolver-7kqxl\" (UID: \"f1a6d6ed-76dd-45b7-bfc7-65f0babd3893\") " pod="openshift-dns/node-resolver-7kqxl"
Jul 07 23:13:37.872443 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872420    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzctr\" (UniqueName: \"kubernetes.io/projected/1e8e98a3-610a-463f-8546-0af03fe1f4f3-kube-api-access-hzctr\") pod \"iptables-alerter-tlptd\" (UID: \"1e8e98a3-610a-463f-8546-0af03fe1f4f3\") " pod="openshift-network-operator/iptables-alerter-tlptd"
Jul 07 23:13:37.872826 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872445    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-run-netns\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.872826 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872485    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/af219d19-df59-4d6d-97a2-8985f717797c-ovn-node-metrics-cert\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.872826 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872508    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-system-cni-dir\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.872826 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-cni-netd\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.872826 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872582    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfz9z\" (UniqueName: \"kubernetes.io/projected/af219d19-df59-4d6d-97a2-8985f717797c-kube-api-access-bfz9z\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.872826 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872627    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-daemon-config\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.872826 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872660    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-kubernetes\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.872826 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872686    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.873214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872858    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/337b8d66-22e8-4cee-a345-c5dd54dca37b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.873214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872866    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.873214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872866    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:37.873214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.872920    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2pxb\" (UniqueName: \"kubernetes.io/projected/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-kube-api-access-q2pxb\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:37.873214 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:37.872975    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:37.873214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873037    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/337b8d66-22e8-4cee-a345-c5dd54dca37b-cni-binary-copy\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.873214 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:37.873073    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs podName:03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b nodeName:}" failed. No retries permitted until 2026-07-07 23:13:38.373029912 +0000 UTC m=+2.958599254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs") pod "network-metrics-daemon-mr5rb" (UID: "03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:37.873214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873175    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-run-ovn-kubernetes\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.873214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873207    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/af219d19-df59-4d6d-97a2-8985f717797c-ovnkube-config\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873257    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-conf-dir\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873306    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-node-log\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-systemd-units\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873365    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-sysconfig\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-systemd\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873437    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-registration-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c342d1b2-9987-404d-88ae-197de63d7d04-host\") pod \"node-ca-g7tzd\" (UID: \"c342d1b2-9987-404d-88ae-197de63d7d04\") " pod="openshift-image-registry/node-ca-g7tzd"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873483    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-kubelet\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-var-lib-openvswitch\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddd72\" (UniqueName: \"kubernetes.io/projected/c342d1b2-9987-404d-88ae-197de63d7d04-kube-api-access-ddd72\") pod \"node-ca-g7tzd\" (UID: \"c342d1b2-9987-404d-88ae-197de63d7d04\") " pod="openshift-image-registry/node-ca-g7tzd"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873555    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-dbus\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873590    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/337b8d66-22e8-4cee-a345-c5dd54dca37b-cni-binary-copy\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/622e9231-8a5d-4998-a298-d76e51084e83-konnectivity-ca\") pod \"konnectivity-agent-wv855\" (UID: \"622e9231-8a5d-4998-a298-d76e51084e83\") " pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:13:37.873674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873675    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-device-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-sys-fs\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873728    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-run-systemd\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873758    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/337b8d66-22e8-4cee-a345-c5dd54dca37b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873784    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e8e98a3-610a-463f-8546-0af03fe1f4f3-iptables-alerter-script\") pod \"iptables-alerter-tlptd\" (UID: \"1e8e98a3-610a-463f-8546-0af03fe1f4f3\") " pod="openshift-network-operator/iptables-alerter-tlptd"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873825    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873851    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-var-lib-kubelet\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873874    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-sysctl-d\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/622e9231-8a5d-4998-a298-d76e51084e83-agent-certs\") pod \"konnectivity-agent-wv855\" (UID: \"622e9231-8a5d-4998-a298-d76e51084e83\") " pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873921    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873950    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-system-cni-dir\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873973    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99cf80ee-8d7c-47a0-b546-13487e27e5cf-cni-binary-copy\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.873996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-run-k8s-cni-cncf-io\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.874040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-etc-kubernetes\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.874063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-sys\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.874326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.874303    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/337b8d66-22e8-4cee-a345-c5dd54dca37b-system-cni-dir\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.874999 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.874340    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/337b8d66-22e8-4cee-a345-c5dd54dca37b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.874999 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.874648    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e8e98a3-610a-463f-8546-0af03fe1f4f3-iptables-alerter-script\") pod \"iptables-alerter-tlptd\" (UID: \"1e8e98a3-610a-463f-8546-0af03fe1f4f3\") " pod="openshift-network-operator/iptables-alerter-tlptd"
Jul 07 23:13:37.877905 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.877876    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 23:13:37.881717 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.881677    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz772\" (UniqueName: \"kubernetes.io/projected/337b8d66-22e8-4cee-a345-c5dd54dca37b-kube-api-access-pz772\") pod \"multus-additional-cni-plugins-5lxn5\" (UID: \"337b8d66-22e8-4cee-a345-c5dd54dca37b\") " pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:37.882367 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.882344    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2pxb\" (UniqueName: \"kubernetes.io/projected/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-kube-api-access-q2pxb\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:37.882519 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.882494    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzctr\" (UniqueName: \"kubernetes.io/projected/1e8e98a3-610a-463f-8546-0af03fe1f4f3-kube-api-access-hzctr\") pod \"iptables-alerter-tlptd\" (UID: \"1e8e98a3-610a-463f-8546-0af03fe1f4f3\") " pod="openshift-network-operator/iptables-alerter-tlptd"
Jul 07 23:13:37.962647 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.962519    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-77.ec2.internal" event={"ID":"f393c9727e900c86fef1db62fb4e666a","Type":"ContainerStarted","Data":"cb0e8f96c96dc998b2358449051d77e55a6df2dd4de1f08763b27db7f16d7f94"}
Jul 07 23:13:37.964193 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.964142    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal" event={"ID":"d0b20c97249348f78521eef655be2a93","Type":"ContainerStarted","Data":"ca40ba1e44d2280430a765e69d8fae6f5013e6a6fa57026ee24cfc3d5ea1ef92"}
Jul 07 23:13:37.974368 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974331    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.974516 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974376    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-var-lib-kubelet\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.974516 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974394    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-sysctl-d\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.974516 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/622e9231-8a5d-4998-a298-d76e51084e83-agent-certs\") pod \"konnectivity-agent-wv855\" (UID: \"622e9231-8a5d-4998-a298-d76e51084e83\") " pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:13:37.974516 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.974516 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974460    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.974516 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974483    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99cf80ee-8d7c-47a0-b546-13487e27e5cf-cni-binary-copy\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.974516 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974506    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-run-k8s-cni-cncf-io\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-etc-kubernetes\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974600    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-sys\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974619    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-sysctl-d\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974623    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-lib-modules\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974681    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqmcz\" (UniqueName: \"kubernetes.io/projected/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-kube-api-access-qqmcz\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974706    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-kubelet-config\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974726    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-lib-modules\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974759    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-socket-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974774    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-etc-kubernetes\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-etc-selinux\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974817    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z84lc\" (UniqueName: \"kubernetes.io/projected/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-kube-api-access-z84lc\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974844    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c342d1b2-9987-404d-88ae-197de63d7d04-serviceca\") pod \"node-ca-g7tzd\" (UID: \"c342d1b2-9987-404d-88ae-197de63d7d04\") " pod="openshift-image-registry/node-ca-g7tzd"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974863    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-etc-selinux\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974868    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-run-openvswitch\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.974897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974900    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-run-openvswitch\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974938    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-var-lib-kubelet\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975167    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99cf80ee-8d7c-47a0-b546-13487e27e5cf-cni-binary-copy\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975212    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-kubelet-config\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975231    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975266    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-run-k8s-cni-cncf-io\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975308    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-socket-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975312    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-sys\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.974900    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/af219d19-df59-4d6d-97a2-8985f717797c-env-overrides\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:37.975324    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975339    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/af219d19-df59-4d6d-97a2-8985f717797c-env-overrides\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975348    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-var-lib-cni-bin\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:37.975383    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret podName:5f0cc873-6ed5-478d-b3b9-f563d8f8b773 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:38.475365967 +0000 UTC m=+3.060935324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret") pod "global-pull-secret-syncer-5kxrk" (UID: "5f0cc873-6ed5-478d-b3b9-f563d8f8b773") : object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975383    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-var-lib-cni-bin\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.975678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-tuned\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975689    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c342d1b2-9987-404d-88ae-197de63d7d04-serviceca\") pod \"node-ca-g7tzd\" (UID: \"c342d1b2-9987-404d-88ae-197de63d7d04\") " pod="openshift-image-registry/node-ca-g7tzd"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.975432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjvqc\" (UniqueName: \"kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc\") pod \"network-check-target-v6x6f\" (UID: \"5785bd92-cfb4-4235-9d78-47c6884049c6\") " pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976116    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc82q\" (UniqueName: \"kubernetes.io/projected/99cf80ee-8d7c-47a0-b546-13487e27e5cf-kube-api-access-pc82q\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976139    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-modprobe-d\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976163    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/af219d19-df59-4d6d-97a2-8985f717797c-ovnkube-script-lib\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-cni-dir\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976207    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-cnibin\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976229    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-hostroot\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976251    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-run\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976272    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-var-lib-kubelet\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-log-socket\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.976330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976318    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-run-ovn\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-run-netns\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976367    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-run-multus-certs\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976387    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-tmp\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9xjj\" (UniqueName: \"kubernetes.io/projected/f1a6d6ed-76dd-45b7-bfc7-65f0babd3893-kube-api-access-x9xjj\") pod \"node-resolver-7kqxl\" (UID: \"f1a6d6ed-76dd-45b7-bfc7-65f0babd3893\") " pod="openshift-dns/node-resolver-7kqxl"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-cni-bin\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976454    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-os-release\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976463    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-run\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976527    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-log-socket\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976531    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-socket-dir-parent\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976589    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-var-lib-kubelet\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976590    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-run-netns\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976624    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-run-multus-certs\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.976955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976705    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-modprobe-d\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.977555 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977421    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/af219d19-df59-4d6d-97a2-8985f717797c-ovnkube-script-lib\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.977555 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977504    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-cni-dir\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.977555 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977577    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-cnibin\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.977745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977625    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-hostroot\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.977745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.976476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-socket-dir-parent\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.977745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-var-lib-cni-multus\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.977745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1a6d6ed-76dd-45b7-bfc7-65f0babd3893-tmp-dir\") pod \"node-resolver-7kqxl\" (UID: \"f1a6d6ed-76dd-45b7-bfc7-65f0babd3893\") " pod="openshift-dns/node-resolver-7kqxl"
Jul 07 23:13:37.977745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-slash\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.977745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977738    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-etc-openvswitch\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978032 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-sysctl-conf\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.978032 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977783    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-host\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.978032 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977806    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f1a6d6ed-76dd-45b7-bfc7-65f0babd3893-hosts-file\") pod \"node-resolver-7kqxl\" (UID: \"f1a6d6ed-76dd-45b7-bfc7-65f0babd3893\") " pod="openshift-dns/node-resolver-7kqxl"
Jul 07 23:13:37.978032 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977831    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-run-netns\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978032 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/af219d19-df59-4d6d-97a2-8985f717797c-ovn-node-metrics-cert\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978032 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977880    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-system-cni-dir\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.978032 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977903    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-cni-netd\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978032 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977928    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfz9z\" (UniqueName: \"kubernetes.io/projected/af219d19-df59-4d6d-97a2-8985f717797c-kube-api-access-bfz9z\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978032 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977953    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-daemon-config\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.978032 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.977975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-kubernetes\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.978032 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-run-ovn-kubernetes\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978043    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/af219d19-df59-4d6d-97a2-8985f717797c-ovnkube-config\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-conf-dir\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978094    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-node-log\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-systemd-units\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978138    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-sysconfig\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978162    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-systemd\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-registration-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c342d1b2-9987-404d-88ae-197de63d7d04-host\") pod \"node-ca-g7tzd\" (UID: \"c342d1b2-9987-404d-88ae-197de63d7d04\") " pod="openshift-image-registry/node-ca-g7tzd"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-kubelet\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978281    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-var-lib-openvswitch\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978309    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddd72\" (UniqueName: \"kubernetes.io/projected/c342d1b2-9987-404d-88ae-197de63d7d04-kube-api-access-ddd72\") pod \"node-ca-g7tzd\" (UID: \"c342d1b2-9987-404d-88ae-197de63d7d04\") " pod="openshift-image-registry/node-ca-g7tzd"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978337    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-dbus\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/622e9231-8a5d-4998-a298-d76e51084e83-konnectivity-ca\") pod \"konnectivity-agent-wv855\" (UID: \"622e9231-8a5d-4998-a298-d76e51084e83\") " pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-device-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-sys-fs\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-run-systemd\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.978546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978526    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-run-systemd\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978590    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-run-ovn\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978632    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-host-var-lib-cni-multus\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978875    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1a6d6ed-76dd-45b7-bfc7-65f0babd3893-tmp-dir\") pod \"node-resolver-7kqxl\" (UID: \"f1a6d6ed-76dd-45b7-bfc7-65f0babd3893\") " pod="openshift-dns/node-resolver-7kqxl"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978944    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-slash\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.978983    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-etc-openvswitch\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979061    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-tmp\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979104    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-sysctl-conf\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979148    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-host\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979188    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f1a6d6ed-76dd-45b7-bfc7-65f0babd3893-hosts-file\") pod \"node-resolver-7kqxl\" (UID: \"f1a6d6ed-76dd-45b7-bfc7-65f0babd3893\") " pod="openshift-dns/node-resolver-7kqxl"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979224    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-run-netns\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979315    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-system-cni-dir\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979357    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-cni-netd\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.979435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979398    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-tuned\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.980078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979485    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-var-lib-openvswitch\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.980078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979540    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-systemd\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.980078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979609    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-kubernetes\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.980078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979646    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-run-ovn-kubernetes\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.980078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979645    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c342d1b2-9987-404d-88ae-197de63d7d04-host\") pod \"node-ca-g7tzd\" (UID: \"c342d1b2-9987-404d-88ae-197de63d7d04\") " pod="openshift-image-registry/node-ca-g7tzd"
Jul 07 23:13:37.980078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979698    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-conf-dir\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.980078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979739    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-node-log\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.980078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979777    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-systemd-units\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.980078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.979818    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-kubelet\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.980078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.980046    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99cf80ee-8d7c-47a0-b546-13487e27e5cf-multus-daemon-config\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.980527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.980110    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/af219d19-df59-4d6d-97a2-8985f717797c-ovnkube-config\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.980527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.980119    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-registration-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.980527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.980190    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99cf80ee-8d7c-47a0-b546-13487e27e5cf-os-release\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.980527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.980238    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-etc-sysconfig\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.980527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.980277    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/622e9231-8a5d-4998-a298-d76e51084e83-konnectivity-ca\") pod \"konnectivity-agent-wv855\" (UID: \"622e9231-8a5d-4998-a298-d76e51084e83\") " pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:13:37.980527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.980331    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-device-dir\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.980527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.980359    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-sys-fs\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.980527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.980361    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-dbus\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:37.980527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.980479    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af219d19-df59-4d6d-97a2-8985f717797c-host-cni-bin\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.980981 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.980759    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/622e9231-8a5d-4998-a298-d76e51084e83-agent-certs\") pod \"konnectivity-agent-wv855\" (UID: \"622e9231-8a5d-4998-a298-d76e51084e83\") " pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:13:37.982738 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.982712    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/af219d19-df59-4d6d-97a2-8985f717797c-ovn-node-metrics-cert\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.986456 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:37.986430    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 23:13:37.986587 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:37.986466    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 23:13:37.986587 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:37.986480    2578 projected.go:196] Error preparing data for projected volume kube-api-access-qjvqc for pod openshift-network-diagnostics/network-check-target-v6x6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:37.986587 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:37.986576    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc podName:5785bd92-cfb4-4235-9d78-47c6884049c6 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:38.486540267 +0000 UTC m=+3.072109620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qjvqc" (UniqueName: "kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc") pod "network-check-target-v6x6f" (UID: "5785bd92-cfb4-4235-9d78-47c6884049c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:37.990926 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.988459    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqmcz\" (UniqueName: \"kubernetes.io/projected/b34e1255-c5ff-4508-a7e1-4f28d65e1b28-kube-api-access-qqmcz\") pod \"tuned-6mqgz\" (UID: \"b34e1255-c5ff-4508-a7e1-4f28d65e1b28\") " pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:37.992504 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.992439    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc82q\" (UniqueName: \"kubernetes.io/projected/99cf80ee-8d7c-47a0-b546-13487e27e5cf-kube-api-access-pc82q\") pod \"multus-fjbhl\" (UID: \"99cf80ee-8d7c-47a0-b546-13487e27e5cf\") " pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:37.993285 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.993121    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z84lc\" (UniqueName: \"kubernetes.io/projected/53a444e6-38fe-40a1-b6f8-234fbfb3e76e-kube-api-access-z84lc\") pod \"aws-ebs-csi-driver-node-96m42\" (UID: \"53a444e6-38fe-40a1-b6f8-234fbfb3e76e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:37.993741 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.993718    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9xjj\" (UniqueName: \"kubernetes.io/projected/f1a6d6ed-76dd-45b7-bfc7-65f0babd3893-kube-api-access-x9xjj\") pod \"node-resolver-7kqxl\" (UID: \"f1a6d6ed-76dd-45b7-bfc7-65f0babd3893\") " pod="openshift-dns/node-resolver-7kqxl"
Jul 07 23:13:37.994344 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.994321    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfz9z\" (UniqueName: \"kubernetes.io/projected/af219d19-df59-4d6d-97a2-8985f717797c-kube-api-access-bfz9z\") pod \"ovnkube-node-57lng\" (UID: \"af219d19-df59-4d6d-97a2-8985f717797c\") " pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:37.995125 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:37.995103    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddd72\" (UniqueName: \"kubernetes.io/projected/c342d1b2-9987-404d-88ae-197de63d7d04-kube-api-access-ddd72\") pod \"node-ca-g7tzd\" (UID: \"c342d1b2-9987-404d-88ae-197de63d7d04\") " pod="openshift-image-registry/node-ca-g7tzd"
Jul 07 23:13:38.074894 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.074865    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tlptd"
Jul 07 23:13:38.084946 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.084918    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5lxn5"
Jul 07 23:13:38.093926 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.093896    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:13:38.104766 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.104734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:13:38.120604 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.120572    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42"
Jul 07 23:13:38.133555 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.133518    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7kqxl"
Jul 07 23:13:38.142397 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.142371    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g7tzd"
Jul 07 23:13:38.147374 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.147350    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fjbhl"
Jul 07 23:13:38.153149 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.153129    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6mqgz"
Jul 07 23:13:38.381376 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.381287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:38.381531 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:38.381441    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:38.381531 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:38.381507    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs podName:03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b nodeName:}" failed. No retries permitted until 2026-07-07 23:13:39.381491704 +0000 UTC m=+3.967061043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs") pod "network-metrics-daemon-mr5rb" (UID: "03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:38.481776 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.481741    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:38.481959 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:38.481888    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:38.482032 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:38.481975    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret podName:5f0cc873-6ed5-478d-b3b9-f563d8f8b773 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:39.481958743 +0000 UTC m=+4.067528082 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret") pod "global-pull-secret-syncer-5kxrk" (UID: "5f0cc873-6ed5-478d-b3b9-f563d8f8b773") : object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:38.582877 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.582803    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjvqc\" (UniqueName: \"kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc\") pod \"network-check-target-v6x6f\" (UID: \"5785bd92-cfb4-4235-9d78-47c6884049c6\") " pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:38.583027 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:38.582963    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 23:13:38.583027 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:38.582989    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 23:13:38.583027 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:38.583004    2578 projected.go:196] Error preparing data for projected volume kube-api-access-qjvqc for pod openshift-network-diagnostics/network-check-target-v6x6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:38.583173 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:38.583069    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc podName:5785bd92-cfb4-4235-9d78-47c6884049c6 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:39.583048423 +0000 UTC m=+4.168617765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjvqc" (UniqueName: "kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc") pod "network-check-target-v6x6f" (UID: "5785bd92-cfb4-4235-9d78-47c6884049c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:38.819001 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.818865    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 23:08:37 +0000 UTC" deadline="2027-12-11 07:56:30.120344281 +0000 UTC"
Jul 07 23:13:38.819001 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:38.818941    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12512h42m51.301407019s"
Jul 07 23:13:39.001267 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:39.001125    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc342d1b2_9987_404d_88ae_197de63d7d04.slice/crio-e63f072bf24ef5ceb95acd2a52bb7de491b8ab4e57b5c7107c4abd54dfad8ea2 WatchSource:0}: Error finding container e63f072bf24ef5ceb95acd2a52bb7de491b8ab4e57b5c7107c4abd54dfad8ea2: Status 404 returned error can't find the container with id e63f072bf24ef5ceb95acd2a52bb7de491b8ab4e57b5c7107c4abd54dfad8ea2
Jul 07 23:13:39.006451 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:39.006424    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337b8d66_22e8_4cee_a345_c5dd54dca37b.slice/crio-b834c2ab7db25eac95b7a3023ad8609ba27cbc6ec28dfe897c2917e6e4b61f46 WatchSource:0}: Error finding container b834c2ab7db25eac95b7a3023ad8609ba27cbc6ec28dfe897c2917e6e4b61f46: Status 404 returned error can't find the container with id b834c2ab7db25eac95b7a3023ad8609ba27cbc6ec28dfe897c2917e6e4b61f46
Jul 07 23:13:39.008498 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:39.008473    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod622e9231_8a5d_4998_a298_d76e51084e83.slice/crio-b858743763506d93acd5e504f969e41621d44c3f1dc614e6963fedf1c13b1c7d WatchSource:0}: Error finding container b858743763506d93acd5e504f969e41621d44c3f1dc614e6963fedf1c13b1c7d: Status 404 returned error can't find the container with id b858743763506d93acd5e504f969e41621d44c3f1dc614e6963fedf1c13b1c7d
Jul 07 23:13:39.011700 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:39.009432    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34e1255_c5ff_4508_a7e1_4f28d65e1b28.slice/crio-4e9d3cae5ce82e7aeb83b92199c7b409d95d095f13dcef03f540c7a008d93a49 WatchSource:0}: Error finding container 4e9d3cae5ce82e7aeb83b92199c7b409d95d095f13dcef03f540c7a008d93a49: Status 404 returned error can't find the container with id 4e9d3cae5ce82e7aeb83b92199c7b409d95d095f13dcef03f540c7a008d93a49
Jul 07 23:13:39.011700 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:39.010476    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a444e6_38fe_40a1_b6f8_234fbfb3e76e.slice/crio-ec6382bee48c8bf334a9132c3fe59e375d6857819b82c2ab522ab49c40c2da92 WatchSource:0}: Error finding container ec6382bee48c8bf334a9132c3fe59e375d6857819b82c2ab522ab49c40c2da92: Status 404 returned error can't find the container with id ec6382bee48c8bf334a9132c3fe59e375d6857819b82c2ab522ab49c40c2da92
Jul 07 23:13:39.011700 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:39.011224    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cf80ee_8d7c_47a0_b546_13487e27e5cf.slice/crio-582e19ecdfb6c0d1297d78308b4f400ee09339a7aabd0f676b4eb83a753d09af WatchSource:0}: Error finding container 582e19ecdfb6c0d1297d78308b4f400ee09339a7aabd0f676b4eb83a753d09af: Status 404 returned error can't find the container with id 582e19ecdfb6c0d1297d78308b4f400ee09339a7aabd0f676b4eb83a753d09af
Jul 07 23:13:39.012482 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:39.012451    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf219d19_df59_4d6d_97a2_8985f717797c.slice/crio-59554d4c3dab08a65eaf3ae24d633ec241a3f780c60763a7eb16cef591bf4d0a WatchSource:0}: Error finding container 59554d4c3dab08a65eaf3ae24d633ec241a3f780c60763a7eb16cef591bf4d0a: Status 404 returned error can't find the container with id 59554d4c3dab08a65eaf3ae24d633ec241a3f780c60763a7eb16cef591bf4d0a
Jul 07 23:13:39.013487 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:39.013381    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a6d6ed_76dd_45b7_bfc7_65f0babd3893.slice/crio-ae5d40a8109837eed6e12ca8009433eb62abaa37a319fd7a6b703f04f4d4c840 WatchSource:0}: Error finding container ae5d40a8109837eed6e12ca8009433eb62abaa37a319fd7a6b703f04f4d4c840: Status 404 returned error can't find the container with id ae5d40a8109837eed6e12ca8009433eb62abaa37a319fd7a6b703f04f4d4c840
Jul 07 23:13:39.014398 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:13:39.014377    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e8e98a3_610a_463f_8546_0af03fe1f4f3.slice/crio-f8894c4334124e9fd6daea4e0234943c036dfd7e18b12d4898a55b15e4e519e1 WatchSource:0}: Error finding container f8894c4334124e9fd6daea4e0234943c036dfd7e18b12d4898a55b15e4e519e1: Status 404 returned error can't find the container with id f8894c4334124e9fd6daea4e0234943c036dfd7e18b12d4898a55b15e4e519e1
Jul 07 23:13:39.391009 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:39.390702    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:39.391165 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:39.390883    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:39.391165 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:39.391145    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs podName:03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b nodeName:}" failed. No retries permitted until 2026-07-07 23:13:41.391073176 +0000 UTC m=+5.976642540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs") pod "network-metrics-daemon-mr5rb" (UID: "03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:39.414046 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:39.414012    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 23:13:39.492138 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:39.492100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:39.492305 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:39.492289    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:39.492378 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:39.492358    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret podName:5f0cc873-6ed5-478d-b3b9-f563d8f8b773 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:41.492338126 +0000 UTC m=+6.077907479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret") pod "global-pull-secret-syncer-5kxrk" (UID: "5f0cc873-6ed5-478d-b3b9-f563d8f8b773") : object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:39.593337 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:39.593296    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjvqc\" (UniqueName: \"kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc\") pod \"network-check-target-v6x6f\" (UID: \"5785bd92-cfb4-4235-9d78-47c6884049c6\") " pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:39.593539 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:39.593516    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 23:13:39.593646 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:39.593546    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 23:13:39.593646 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:39.593574    2578 projected.go:196] Error preparing data for projected volume kube-api-access-qjvqc for pod openshift-network-diagnostics/network-check-target-v6x6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:39.593646 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:39.593635    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc podName:5785bd92-cfb4-4235-9d78-47c6884049c6 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:41.593614814 +0000 UTC m=+6.179184154 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjvqc" (UniqueName: "kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc") pod "network-check-target-v6x6f" (UID: "5785bd92-cfb4-4235-9d78-47c6884049c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:39.819617 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:39.819514    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 23:08:37 +0000 UTC" deadline="2028-03-22 04:07:07.304710186 +0000 UTC"
Jul 07 23:13:39.819617 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:39.819551    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14956h53m27.485163989s"
Jul 07 23:13:39.959316 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:39.959282    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:39.959531 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:39.959504    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:39.959998 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:39.959979    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:39.960100 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:39.960077    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:39.960169 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:39.960144    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:39.960233 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:39.960215    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:13:39.991958 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:39.991916    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fjbhl" event={"ID":"99cf80ee-8d7c-47a0-b546-13487e27e5cf","Type":"ContainerStarted","Data":"582e19ecdfb6c0d1297d78308b4f400ee09339a7aabd0f676b4eb83a753d09af"}
Jul 07 23:13:39.997322 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:39.997218    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42" event={"ID":"53a444e6-38fe-40a1-b6f8-234fbfb3e76e","Type":"ContainerStarted","Data":"ec6382bee48c8bf334a9132c3fe59e375d6857819b82c2ab522ab49c40c2da92"}
Jul 07 23:13:40.007804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:40.007763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6mqgz" event={"ID":"b34e1255-c5ff-4508-a7e1-4f28d65e1b28","Type":"ContainerStarted","Data":"4e9d3cae5ce82e7aeb83b92199c7b409d95d095f13dcef03f540c7a008d93a49"}
Jul 07 23:13:40.017723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:40.017684    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxn5" event={"ID":"337b8d66-22e8-4cee-a345-c5dd54dca37b","Type":"ContainerStarted","Data":"b834c2ab7db25eac95b7a3023ad8609ba27cbc6ec28dfe897c2917e6e4b61f46"}
Jul 07 23:13:40.024286 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:40.024248    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g7tzd" event={"ID":"c342d1b2-9987-404d-88ae-197de63d7d04","Type":"ContainerStarted","Data":"e63f072bf24ef5ceb95acd2a52bb7de491b8ab4e57b5c7107c4abd54dfad8ea2"}
Jul 07 23:13:40.032783 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:40.032742    2578 generic.go:356] "Generic (PLEG): container finished" podID="d0b20c97249348f78521eef655be2a93" containerID="0c64ce5ed720ce26bf8a526e7675b96cdfa7fd7a901ac5be775b8f2fa574e8c1" exitCode=0
Jul 07 23:13:40.032956 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:40.032859    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal" event={"ID":"d0b20c97249348f78521eef655be2a93","Type":"ContainerDied","Data":"0c64ce5ed720ce26bf8a526e7675b96cdfa7fd7a901ac5be775b8f2fa574e8c1"}
Jul 07 23:13:40.048644 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:40.044302    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-77.ec2.internal" event={"ID":"f393c9727e900c86fef1db62fb4e666a","Type":"ContainerStarted","Data":"72363c46f4c96f9ed86913f311663a601c534db04386d15966659d5d88b517fb"}
Jul 07 23:13:40.054630 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:40.054597    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wv855" event={"ID":"622e9231-8a5d-4998-a298-d76e51084e83","Type":"ContainerStarted","Data":"b858743763506d93acd5e504f969e41621d44c3f1dc614e6963fedf1c13b1c7d"}
Jul 07 23:13:40.066714 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:40.066667    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tlptd" event={"ID":"1e8e98a3-610a-463f-8546-0af03fe1f4f3","Type":"ContainerStarted","Data":"f8894c4334124e9fd6daea4e0234943c036dfd7e18b12d4898a55b15e4e519e1"}
Jul 07 23:13:40.075821 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:40.075761    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-77.ec2.internal" podStartSLOduration=3.075739334 podStartE2EDuration="3.075739334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:40.075042926 +0000 UTC m=+4.660612288" watchObservedRunningTime="2026-07-07 23:13:40.075739334 +0000 UTC m=+4.661308698"
Jul 07 23:13:40.080973 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:40.080934    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7kqxl" event={"ID":"f1a6d6ed-76dd-45b7-bfc7-65f0babd3893","Type":"ContainerStarted","Data":"ae5d40a8109837eed6e12ca8009433eb62abaa37a319fd7a6b703f04f4d4c840"}
Jul 07 23:13:40.084914 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:40.084801    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-57lng" event={"ID":"af219d19-df59-4d6d-97a2-8985f717797c","Type":"ContainerStarted","Data":"59554d4c3dab08a65eaf3ae24d633ec241a3f780c60763a7eb16cef591bf4d0a"}
Jul 07 23:13:41.097688 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:41.097616    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal" event={"ID":"d0b20c97249348f78521eef655be2a93","Type":"ContainerStarted","Data":"b1aee84b59bd06c041b5121891e1e0e1629484f0854687ed44f8334309ae9071"}
Jul 07 23:13:41.122749 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:41.122692    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-77.ec2.internal" podStartSLOduration=4.122669193 podStartE2EDuration="4.122669193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:41.122386765 +0000 UTC m=+5.707956126" watchObservedRunningTime="2026-07-07 23:13:41.122669193 +0000 UTC m=+5.708238547"
Jul 07 23:13:41.413675 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:41.412915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:41.413675 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:41.413112    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:41.413675 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:41.413177    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs podName:03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b nodeName:}" failed. No retries permitted until 2026-07-07 23:13:45.413158625 +0000 UTC m=+9.998727967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs") pod "network-metrics-daemon-mr5rb" (UID: "03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:41.513986 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:41.513377    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:41.513986 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:41.513534    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:41.513986 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:41.513622    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret podName:5f0cc873-6ed5-478d-b3b9-f563d8f8b773 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:45.513603505 +0000 UTC m=+10.099172847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret") pod "global-pull-secret-syncer-5kxrk" (UID: "5f0cc873-6ed5-478d-b3b9-f563d8f8b773") : object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:41.613922 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:41.613885    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjvqc\" (UniqueName: \"kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc\") pod \"network-check-target-v6x6f\" (UID: \"5785bd92-cfb4-4235-9d78-47c6884049c6\") " pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:41.614096 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:41.614079    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 23:13:41.614154 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:41.614103    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 23:13:41.614154 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:41.614117    2578 projected.go:196] Error preparing data for projected volume kube-api-access-qjvqc for pod openshift-network-diagnostics/network-check-target-v6x6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:41.614244 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:41.614179    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc podName:5785bd92-cfb4-4235-9d78-47c6884049c6 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:45.614160366 +0000 UTC m=+10.199729710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjvqc" (UniqueName: "kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc") pod "network-check-target-v6x6f" (UID: "5785bd92-cfb4-4235-9d78-47c6884049c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:41.957704 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:41.957672    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:41.957875 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:41.957803    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:13:41.958219 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:41.958186    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:41.958323 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:41.958293    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:41.960348 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:41.960325    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:41.960453 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:41.960420    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:43.961432 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:43.961291    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:43.961432 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:43.961320    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:43.961432 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:43.961321    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:43.962017 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:43.961425    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:13:43.962017 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:43.961663    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:43.962017 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:43.961745    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:45.446901 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:45.446856    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:45.447297 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:45.447027    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:45.447297 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:45.447099    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs podName:03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b nodeName:}" failed. No retries permitted until 2026-07-07 23:13:53.447077785 +0000 UTC m=+18.032647127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs") pod "network-metrics-daemon-mr5rb" (UID: "03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:45.547630 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:45.547575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:45.547832 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:45.547698    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:45.547832 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:45.547785    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret podName:5f0cc873-6ed5-478d-b3b9-f563d8f8b773 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:53.547763577 +0000 UTC m=+18.133332919 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret") pod "global-pull-secret-syncer-5kxrk" (UID: "5f0cc873-6ed5-478d-b3b9-f563d8f8b773") : object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:45.648603 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:45.648549    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjvqc\" (UniqueName: \"kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc\") pod \"network-check-target-v6x6f\" (UID: \"5785bd92-cfb4-4235-9d78-47c6884049c6\") " pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:45.648770 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:45.648729    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 23:13:45.648770 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:45.648755    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 23:13:45.648770 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:45.648772    2578 projected.go:196] Error preparing data for projected volume kube-api-access-qjvqc for pod openshift-network-diagnostics/network-check-target-v6x6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:45.648920 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:45.648837    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc podName:5785bd92-cfb4-4235-9d78-47c6884049c6 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:53.648819548 +0000 UTC m=+18.234388893 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjvqc" (UniqueName: "kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc") pod "network-check-target-v6x6f" (UID: "5785bd92-cfb4-4235-9d78-47c6884049c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:45.961228 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:45.960779    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:45.961228 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:45.960895    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:13:45.961228 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:45.961011    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:45.961228 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:45.961013    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:45.961228 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:45.961110    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:45.961228 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:45.961196    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:47.959676 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:47.959645    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:47.960108 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:47.959687    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:47.960108 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:47.959752    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:47.960108 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:47.959766    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:47.960108 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:47.959845    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:47.960108 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:47.959953    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:13:49.956664 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:49.956628    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:49.957090 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:49.956628    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:49.957090 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:49.956762    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:49.957090 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:49.956628    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:49.957090 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:49.956834    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:13:49.957090 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:49.956929    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:51.957344 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:51.957307    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:51.957344 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:51.957343    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:51.957945 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:51.957313    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:51.957945 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:51.957452    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:13:51.957945 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:51.957582    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:51.957945 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:51.957686    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:53.508704 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:53.508658    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:53.509184 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:53.508806    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:53.509184 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:53.508887    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs podName:03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b nodeName:}" failed. No retries permitted until 2026-07-07 23:14:09.508865104 +0000 UTC m=+34.094434464 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs") pod "network-metrics-daemon-mr5rb" (UID: "03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:13:53.609199 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:53.609156    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:53.609385 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:53.609340    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:53.609442 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:53.609426    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret podName:5f0cc873-6ed5-478d-b3b9-f563d8f8b773 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:09.609403206 +0000 UTC m=+34.194972564 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret") pod "global-pull-secret-syncer-5kxrk" (UID: "5f0cc873-6ed5-478d-b3b9-f563d8f8b773") : object "kube-system"/"original-pull-secret" not registered
Jul 07 23:13:53.710526 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:53.710489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjvqc\" (UniqueName: \"kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc\") pod \"network-check-target-v6x6f\" (UID: \"5785bd92-cfb4-4235-9d78-47c6884049c6\") " pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:53.710738 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:53.710635    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 23:13:53.710738 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:53.710655    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 23:13:53.710738 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:53.710667    2578 projected.go:196] Error preparing data for projected volume kube-api-access-qjvqc for pod openshift-network-diagnostics/network-check-target-v6x6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:53.710738 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:53.710725    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc podName:5785bd92-cfb4-4235-9d78-47c6884049c6 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:09.710711385 +0000 UTC m=+34.296280724 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjvqc" (UniqueName: "kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc") pod "network-check-target-v6x6f" (UID: "5785bd92-cfb4-4235-9d78-47c6884049c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:13:53.956676 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:53.956636    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:53.956676 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:53.956658    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:53.956940 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:53.956636    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:53.956940 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:53.956783    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:53.956940 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:53.956846    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:53.956940 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:53.956882    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:13:55.957446 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:55.957408    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:55.957901 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:55.957523    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:55.957901 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:55.957600    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:55.957901 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:55.957678    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:55.957901 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:55.957701    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:55.957901 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:55.957811    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:13:57.957505 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:57.957268    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:57.958141 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:57.957274    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:57.958141 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:57.957547    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:57.958141 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:57.957636    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:57.958141 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:57.957274    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:57.958141 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:57.957716    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:13:58.137391 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.137354    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6mqgz" event={"ID":"b34e1255-c5ff-4508-a7e1-4f28d65e1b28","Type":"ContainerStarted","Data":"7034810f672e0716f9e44fb1899be59cea2d5b3f5585b996cf1a74887f271e9f"}
Jul 07 23:13:58.138994 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.138964    2578 generic.go:356] "Generic (PLEG): container finished" podID="337b8d66-22e8-4cee-a345-c5dd54dca37b" containerID="c852a503894538c45c2f8438d506553bc1f7ec3919228708b5baf31067974013" exitCode=0
Jul 07 23:13:58.139096 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.139045    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxn5" event={"ID":"337b8d66-22e8-4cee-a345-c5dd54dca37b","Type":"ContainerDied","Data":"c852a503894538c45c2f8438d506553bc1f7ec3919228708b5baf31067974013"}
Jul 07 23:13:58.140803 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.140744    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g7tzd" event={"ID":"c342d1b2-9987-404d-88ae-197de63d7d04","Type":"ContainerStarted","Data":"59f24065eede63f7e0b49bfdf6d850bc530d26dd438e6e65bea13447ff22f3ed"}
Jul 07 23:13:58.145050 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.145020    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wv855" event={"ID":"622e9231-8a5d-4998-a298-d76e51084e83","Type":"ContainerStarted","Data":"fc942af7de3f602eef403fea22c57a96dd0a1813b87a2a693176db985247d330"}
Jul 07 23:13:58.147308 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.147280    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7kqxl" event={"ID":"f1a6d6ed-76dd-45b7-bfc7-65f0babd3893","Type":"ContainerStarted","Data":"dda2d81ef7ebc576b04a85a6bf58bcb1754386f76922f87e5039930378c71ed0"}
Jul 07 23:13:58.149742 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.149685    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-57lng" event={"ID":"af219d19-df59-4d6d-97a2-8985f717797c","Type":"ContainerStarted","Data":"2b934e5fcead43ae5f369d59bf9a8d6b1963faf0aebaf85de0c067a8d70fb99d"}
Jul 07 23:13:58.149811 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.149760    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-57lng" event={"ID":"af219d19-df59-4d6d-97a2-8985f717797c","Type":"ContainerStarted","Data":"bd57ae82a931403f98c4f50c4cb1f1648fa6da938fc48136abf9e7d4715baaf7"}
Jul 07 23:13:58.149873 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.149812    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-57lng" event={"ID":"af219d19-df59-4d6d-97a2-8985f717797c","Type":"ContainerStarted","Data":"e24faf481c7a5b7789d8aa5a4a190c66a97146c4ed21ddad0ce706755465895e"}
Jul 07 23:13:58.149873 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.149859    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-57lng" event={"ID":"af219d19-df59-4d6d-97a2-8985f717797c","Type":"ContainerStarted","Data":"06292ae6f8908bc54e31ecaa15ab3100eca18ec85e002b8e156565be4b20d76a"}
Jul 07 23:13:58.151447 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.151403    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fjbhl" event={"ID":"99cf80ee-8d7c-47a0-b546-13487e27e5cf","Type":"ContainerStarted","Data":"8b30d66ec7f233d07624dbb68db61a8b1394c9f98d241d2563ef2734e11165de"}
Jul 07 23:13:58.153211 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.153191    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42" event={"ID":"53a444e6-38fe-40a1-b6f8-234fbfb3e76e","Type":"ContainerStarted","Data":"2eb09ee1a8c30404cce4ef9e1c4d7a7a9a8fe0fbdbc08428578f21266cc08c81"}
Jul 07 23:13:58.192769 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.192719    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6mqgz" podStartSLOduration=3.6865898550000002 podStartE2EDuration="22.192697375s" totalImagesPullingTime="18.50610752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:58.170674926 +0000 UTC m=+22.756244289" watchObservedRunningTime="2026-07-07 23:13:58.192697375 +0000 UTC m=+22.778266737"
Jul 07 23:13:58.219402 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.219304    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wv855" podStartSLOduration=8.341636129 podStartE2EDuration="22.219282789s" totalImagesPullingTime="13.87764666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:58.19296711 +0000 UTC m=+22.778536471" watchObservedRunningTime="2026-07-07 23:13:58.219282789 +0000 UTC m=+22.804852150"
Jul 07 23:13:58.219976 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.219946    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7kqxl" podStartSLOduration=3.740203654 podStartE2EDuration="22.2199387s" totalImagesPullingTime="18.479735046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:58.219518747 +0000 UTC m=+22.805088110" watchObservedRunningTime="2026-07-07 23:13:58.2199387 +0000 UTC m=+22.805508061"
Jul 07 23:13:58.300487 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.300436    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g7tzd" podStartSLOduration=3.788683125 podStartE2EDuration="22.3004199s" totalImagesPullingTime="18.511736775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:58.300141683 +0000 UTC m=+22.885711045" watchObservedRunningTime="2026-07-07 23:13:58.3004199 +0000 UTC m=+22.885989261"
Jul 07 23:13:58.336647 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.336592    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fjbhl" podStartSLOduration=3.764619304 podStartE2EDuration="22.336575273s" totalImagesPullingTime="18.571955969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:58.335309551 +0000 UTC m=+22.920878912" watchObservedRunningTime="2026-07-07 23:13:58.336575273 +0000 UTC m=+22.922144625"
Jul 07 23:13:58.772733 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.772708    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 23:13:58.836333 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.836188    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T23:13:58.772728207Z","UUID":"e460afac-4f5e-4926-8aa9-0d75820be34f","Handler":null,"Name":"","Endpoint":""}
Jul 07 23:13:58.837892 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.837872    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 23:13:58.838028 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:58.837914    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 23:13:59.157210 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:59.157173    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tlptd" event={"ID":"1e8e98a3-610a-463f-8546-0af03fe1f4f3","Type":"ContainerStarted","Data":"bf55a0e7652c1949d0741edf3d738ea69254ce5a9ea7313cea8a2a6bc7aa0e83"}
Jul 07 23:13:59.160713 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:59.160555    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-57lng" event={"ID":"af219d19-df59-4d6d-97a2-8985f717797c","Type":"ContainerStarted","Data":"261a7175acfa7481b79937bc129438618aa5adfbcbb40590450e01d6981f9189"}
Jul 07 23:13:59.160713 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:59.160652    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-57lng" event={"ID":"af219d19-df59-4d6d-97a2-8985f717797c","Type":"ContainerStarted","Data":"d0ee051b6f89985c73f6ab20a9c483ae01979e3d2d3c6b3ee72614dcde533182"}
Jul 07 23:13:59.166111 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:59.166074    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42" event={"ID":"53a444e6-38fe-40a1-b6f8-234fbfb3e76e","Type":"ContainerStarted","Data":"83c5f921aa459a9ab4cc596c271dce3937ca4666a1857e6a9e21aaa53d98d823"}
Jul 07 23:13:59.180960 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:59.180906    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tlptd" podStartSLOduration=5.70341727 podStartE2EDuration="24.180890896s" totalImagesPullingTime="18.477473626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:59.180777979 +0000 UTC m=+23.766347340" watchObservedRunningTime="2026-07-07 23:13:59.180890896 +0000 UTC m=+23.766460257"
Jul 07 23:13:59.956725 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:59.956688    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:13:59.956882 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:59.956688    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:13:59.956882 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:59.956833    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:13:59.957003 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:13:59.956688    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:13:59.957003 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:59.956895    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:13:59.957003 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:13:59.956986    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:14:00.170744 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:00.170657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42" event={"ID":"53a444e6-38fe-40a1-b6f8-234fbfb3e76e","Type":"ContainerStarted","Data":"687686dabb1a2f795a180d0e646c48ab11bd8e30d3109273bbf49b1a90ac6cca"}
Jul 07 23:14:00.191898 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:00.191843    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-96m42" podStartSLOduration=3.634054123 podStartE2EDuration="24.191821365s" totalImagesPullingTime="20.557767242s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:00.190766334 +0000 UTC m=+24.776335695" watchObservedRunningTime="2026-07-07 23:14:00.191821365 +0000 UTC m=+24.777390721"
Jul 07 23:14:01.175843 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:01.175806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-57lng" event={"ID":"af219d19-df59-4d6d-97a2-8985f717797c","Type":"ContainerStarted","Data":"ac1130778de44807145a26485d3532c645bab8ae66dc35198adec55dad4332ac"}
Jul 07 23:14:01.956862 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:01.956651    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:14:01.957060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:01.956651    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:14:01.957060 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:01.956959    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:14:01.957060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:01.956692    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:14:01.957060 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:01.957052    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:14:01.957280 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:01.957106    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:14:03.040376 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:03.040284    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:14:03.040993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:03.040982    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:14:03.180627 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:03.180595    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:14:03.181361 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:03.181345    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wv855"
Jul 07 23:14:03.957582 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:03.957385    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:14:03.957750 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:03.957385    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:14:03.957750 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:03.957660    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:14:03.957750 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:03.957447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:14:03.957846 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:03.957722    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:14:03.957846 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:03.957833    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:14:04.186792 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:04.186755    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-57lng" event={"ID":"af219d19-df59-4d6d-97a2-8985f717797c","Type":"ContainerStarted","Data":"c29dc5cb18e05ea5585c08d9401f795f2908fb1191fe3d8bfc5fe10a9036d048"}
Jul 07 23:14:04.187731 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:04.186938    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:14:04.187731 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:04.187007    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:14:04.188884 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:04.188854    2578 generic.go:356] "Generic (PLEG): container finished" podID="337b8d66-22e8-4cee-a345-c5dd54dca37b" containerID="9c765c6b8ef130024fae9d32bce9c523969f13be9d85b95ad8a4bb1abf27c596" exitCode=0
Jul 07 23:14:04.189016 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:04.188917    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxn5" event={"ID":"337b8d66-22e8-4cee-a345-c5dd54dca37b","Type":"ContainerDied","Data":"9c765c6b8ef130024fae9d32bce9c523969f13be9d85b95ad8a4bb1abf27c596"}
Jul 07 23:14:04.203589 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:04.203544    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:14:04.261460 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:04.261358    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-57lng" podStartSLOduration=9.723395474 podStartE2EDuration="28.261341897s" totalImagesPullingTime="18.537946423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:04.224045597 +0000 UTC m=+28.809614958" watchObservedRunningTime="2026-07-07 23:14:04.261341897 +0000 UTC m=+28.846911257"
Jul 07 23:14:05.165898 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:05.165853    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5kxrk"]
Jul 07 23:14:05.166093 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:05.166000    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:14:05.166093 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:05.166079    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:14:05.168924 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:05.168817    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v6x6f"]
Jul 07 23:14:05.169056 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:05.168935    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:14:05.169056 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:05.169028    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:14:05.169400 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:05.169379    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mr5rb"]
Jul 07 23:14:05.169488 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:05.169472    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:14:05.169638 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:05.169615    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:14:05.191478 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:05.191449    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:14:05.208581 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:05.208482    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:14:06.194815 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:06.194643    2578 generic.go:356] "Generic (PLEG): container finished" podID="337b8d66-22e8-4cee-a345-c5dd54dca37b" containerID="b0937125bc90aebf04186a232eab16db5470628ae6bc01d9265af1d1ffcedebc" exitCode=0
Jul 07 23:14:06.195275 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:06.194726    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxn5" event={"ID":"337b8d66-22e8-4cee-a345-c5dd54dca37b","Type":"ContainerDied","Data":"b0937125bc90aebf04186a232eab16db5470628ae6bc01d9265af1d1ffcedebc"}
Jul 07 23:14:06.957109 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:06.957067    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:14:06.957290 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:06.957070    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:14:06.957290 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:06.957210    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:14:06.957290 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:06.957215    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:14:06.957437 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:06.957309    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:14:06.957437 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:06.957384    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:14:08.201402 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:08.201365    2578 generic.go:356] "Generic (PLEG): container finished" podID="337b8d66-22e8-4cee-a345-c5dd54dca37b" containerID="c48c07fb07e126d5d27f60a8a11380a92cbf6ffe17689523be3c6bbf8eb6041f" exitCode=0
Jul 07 23:14:08.201937 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:08.201428    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxn5" event={"ID":"337b8d66-22e8-4cee-a345-c5dd54dca37b","Type":"ContainerDied","Data":"c48c07fb07e126d5d27f60a8a11380a92cbf6ffe17689523be3c6bbf8eb6041f"}
Jul 07 23:14:08.956685 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:08.956646    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:14:08.956873 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:08.956646    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:14:08.956873 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:08.956777    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kxrk" podUID="5f0cc873-6ed5-478d-b3b9-f563d8f8b773"
Jul 07 23:14:08.956873 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:08.956654    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:14:08.956873 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:08.956848    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mr5rb" podUID="03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b"
Jul 07 23:14:08.957068 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:08.956913    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6x6f" podUID="5785bd92-cfb4-4235-9d78-47c6884049c6"
Jul 07 23:14:09.532660 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:09.532622    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:14:09.533054 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:09.532821    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:14:09.533054 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:09.532894    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs podName:03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b nodeName:}" failed. No retries permitted until 2026-07-07 23:14:41.532878361 +0000 UTC m=+66.118447719 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs") pod "network-metrics-daemon-mr5rb" (UID: "03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 23:14:09.632999 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:09.632961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:14:09.633172 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:09.633099    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 23:14:09.633172 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:09.633157    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret podName:5f0cc873-6ed5-478d-b3b9-f563d8f8b773 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:41.633141037 +0000 UTC m=+66.218710622 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret") pod "global-pull-secret-syncer-5kxrk" (UID: "5f0cc873-6ed5-478d-b3b9-f563d8f8b773") : object "kube-system"/"original-pull-secret" not registered
Jul 07 23:14:09.733915 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:09.733865    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjvqc\" (UniqueName: \"kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc\") pod \"network-check-target-v6x6f\" (UID: \"5785bd92-cfb4-4235-9d78-47c6884049c6\") " pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:14:09.734113 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:09.734058    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 23:14:09.734113 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:09.734083    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 23:14:09.734113 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:09.734092    2578 projected.go:196] Error preparing data for projected volume kube-api-access-qjvqc for pod openshift-network-diagnostics/network-check-target-v6x6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:14:09.734246 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:09.734157    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc podName:5785bd92-cfb4-4235-9d78-47c6884049c6 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:41.734142131 +0000 UTC m=+66.319711472 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjvqc" (UniqueName: "kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc") pod "network-check-target-v6x6f" (UID: "5785bd92-cfb4-4235-9d78-47c6884049c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 23:14:10.225156 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.225127    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-77.ec2.internal" event="NodeReady"
Jul 07 23:14:10.225343 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.225283    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 23:14:10.299693 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.299606    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7cbhm"]
Jul 07 23:14:10.301629 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.301602    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7cbhm"
Jul 07 23:14:10.306172 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.306042    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 23:14:10.306397 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.306376    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m6fb4"]
Jul 07 23:14:10.306825 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.306805    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mjlml\""
Jul 07 23:14:10.306907 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.306830    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 23:14:10.306907 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.306830    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 23:14:10.308149 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.308131    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.311805 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.311785    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 23:14:10.312624 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.312608    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wxtn5\""
Jul 07 23:14:10.312624 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.312615    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 23:14:10.312934 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.312920    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 23:14:10.312999 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.312955    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 23:14:10.323677 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.323651    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7cbhm"]
Jul 07 23:14:10.344195 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.344157    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m6fb4"]
Jul 07 23:14:10.397636 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.397602    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6vvnp"]
Jul 07 23:14:10.399618 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.399594    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.403101 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.403080    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 23:14:10.403101 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.403095    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 23:14:10.404011 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.403994    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-94qrf\""
Jul 07 23:14:10.414631 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.414602    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6vvnp"]
Jul 07 23:14:10.440102 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.440069    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7731486d-cda9-438c-8852-58ff55acb031-data-volume\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.440102 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.440102    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.440316 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.440140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7731486d-cda9-438c-8852-58ff55acb031-crio-socket\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.440316 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.440163    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.440316 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.440221    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj5vd\" (UniqueName: \"kubernetes.io/projected/7731486d-cda9-438c-8852-58ff55acb031-kube-api-access-fj5vd\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.440316 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.440251    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13d513a6-1f84-482a-985a-80339590cd66-cert\") pod \"ingress-canary-7cbhm\" (UID: \"13d513a6-1f84-482a-985a-80339590cd66\") " pod="openshift-ingress-canary/ingress-canary-7cbhm"
Jul 07 23:14:10.440316 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.440270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zn6h\" (UniqueName: \"kubernetes.io/projected/13d513a6-1f84-482a-985a-80339590cd66-kube-api-access-6zn6h\") pod \"ingress-canary-7cbhm\" (UID: \"13d513a6-1f84-482a-985a-80339590cd66\") " pod="openshift-ingress-canary/ingress-canary-7cbhm"
Jul 07 23:14:10.541600 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.541549    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg24k\" (UniqueName: \"kubernetes.io/projected/3f32a465-635f-48b6-b22a-f4e5d481db0b-kube-api-access-kg24k\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.541672    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.541730    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f32a465-635f-48b6-b22a-f4e5d481db0b-tmp-dir\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.541764    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3f32a465-635f-48b6-b22a-f4e5d481db0b-config-volume\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.541797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj5vd\" (UniqueName: \"kubernetes.io/projected/7731486d-cda9-438c-8852-58ff55acb031-kube-api-access-fj5vd\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.541827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13d513a6-1f84-482a-985a-80339590cd66-cert\") pod \"ingress-canary-7cbhm\" (UID: \"13d513a6-1f84-482a-985a-80339590cd66\") " pod="openshift-ingress-canary/ingress-canary-7cbhm"
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.541855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zn6h\" (UniqueName: \"kubernetes.io/projected/13d513a6-1f84-482a-985a-80339590cd66-kube-api-access-6zn6h\") pod \"ingress-canary-7cbhm\" (UID: \"13d513a6-1f84-482a-985a-80339590cd66\") " pod="openshift-ingress-canary/ingress-canary-7cbhm"
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.541924    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7731486d-cda9-438c-8852-58ff55acb031-data-volume\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.541961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.542021    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7731486d-cda9-438c-8852-58ff55acb031-crio-socket\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.542045    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:10.542079    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:10.542144    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:10.542147    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm podName:7731486d-cda9-438c-8852-58ff55acb031 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:11.042128945 +0000 UTC m=+35.627698299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m6fb4" (UID: "7731486d-cda9-438c-8852-58ff55acb031") : configmap "kube-rbac-proxy" not found
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:10.542207    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls podName:7731486d-cda9-438c-8852-58ff55acb031 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:11.04219202 +0000 UTC m=+35.627761362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m6fb4" (UID: "7731486d-cda9-438c-8852-58ff55acb031") : secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:10.542265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.542230    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7731486d-cda9-438c-8852-58ff55acb031-crio-socket\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.542887 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.542303    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7731486d-cda9-438c-8852-58ff55acb031-data-volume\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.546592 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.546549    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13d513a6-1f84-482a-985a-80339590cd66-cert\") pod \"ingress-canary-7cbhm\" (UID: \"13d513a6-1f84-482a-985a-80339590cd66\") " pod="openshift-ingress-canary/ingress-canary-7cbhm"
Jul 07 23:14:10.551186 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.551120    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj5vd\" (UniqueName: \"kubernetes.io/projected/7731486d-cda9-438c-8852-58ff55acb031-kube-api-access-fj5vd\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:10.551276 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.551214    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zn6h\" (UniqueName: \"kubernetes.io/projected/13d513a6-1f84-482a-985a-80339590cd66-kube-api-access-6zn6h\") pod \"ingress-canary-7cbhm\" (UID: \"13d513a6-1f84-482a-985a-80339590cd66\") " pod="openshift-ingress-canary/ingress-canary-7cbhm"
Jul 07 23:14:10.612396 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.612364    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7cbhm"
Jul 07 23:14:10.643014 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.642976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg24k\" (UniqueName: \"kubernetes.io/projected/3f32a465-635f-48b6-b22a-f4e5d481db0b-kube-api-access-kg24k\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.643179 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.643042    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.643179 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.643075    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f32a465-635f-48b6-b22a-f4e5d481db0b-tmp-dir\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.643179 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.643101    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3f32a465-635f-48b6-b22a-f4e5d481db0b-config-volume\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.643432 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:10.643404    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 23:14:10.643519 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:10.643497    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls podName:3f32a465-635f-48b6-b22a-f4e5d481db0b nodeName:}" failed. No retries permitted until 2026-07-07 23:14:11.143474351 +0000 UTC m=+35.729043694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls") pod "dns-default-6vvnp" (UID: "3f32a465-635f-48b6-b22a-f4e5d481db0b") : secret "dns-default-metrics-tls" not found
Jul 07 23:14:10.643615 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.643543    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f32a465-635f-48b6-b22a-f4e5d481db0b-tmp-dir\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.643855 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.643758    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3f32a465-635f-48b6-b22a-f4e5d481db0b-config-volume\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.655114 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.655084    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg24k\" (UniqueName: \"kubernetes.io/projected/3f32a465-635f-48b6-b22a-f4e5d481db0b-kube-api-access-kg24k\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:10.764609 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.764383    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7cbhm"]
Jul 07 23:14:10.769395 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:10.769361    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d513a6_1f84_482a_985a_80339590cd66.slice/crio-d66c5882196e4363f11aea00481c516072fd1032825ba279bdc17e26ace96bf7 WatchSource:0}: Error finding container d66c5882196e4363f11aea00481c516072fd1032825ba279bdc17e26ace96bf7: Status 404 returned error can't find the container with id d66c5882196e4363f11aea00481c516072fd1032825ba279bdc17e26ace96bf7
Jul 07 23:14:10.777145 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.777123    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7kqxl_f1a6d6ed-76dd-45b7-bfc7-65f0babd3893/dns-node-resolver/0.log"
Jul 07 23:14:10.957352 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.957298    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:14:10.957574 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.957318    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:14:10.957574 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.957314    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:14:10.960746 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.960719    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 23:14:10.960880 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.960757    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 23:14:10.960880 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.960808    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hxkkf\""
Jul 07 23:14:10.960880 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.960732    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 23:14:10.960880 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.960719    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nhfdw\""
Jul 07 23:14:10.961104 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:10.960761    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 23:14:11.047471 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:11.047430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:11.047670 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:11.047491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:11.047670 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:11.047574    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 23:14:11.047670 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:11.047654    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm podName:7731486d-cda9-438c-8852-58ff55acb031 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:12.047637445 +0000 UTC m=+36.633206784 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m6fb4" (UID: "7731486d-cda9-438c-8852-58ff55acb031") : configmap "kube-rbac-proxy" not found
Jul 07 23:14:11.047670 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:11.047656    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:11.047821 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:11.047709    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls podName:7731486d-cda9-438c-8852-58ff55acb031 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:12.047695638 +0000 UTC m=+36.633264980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m6fb4" (UID: "7731486d-cda9-438c-8852-58ff55acb031") : secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:11.148502 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:11.148448    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:11.148704 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:11.148611    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 23:14:11.148704 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:11.148684    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls podName:3f32a465-635f-48b6-b22a-f4e5d481db0b nodeName:}" failed. No retries permitted until 2026-07-07 23:14:12.148664351 +0000 UTC m=+36.734233695 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls") pod "dns-default-6vvnp" (UID: "3f32a465-635f-48b6-b22a-f4e5d481db0b") : secret "dns-default-metrics-tls" not found
Jul 07 23:14:11.209548 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:11.209452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7cbhm" event={"ID":"13d513a6-1f84-482a-985a-80339590cd66","Type":"ContainerStarted","Data":"d66c5882196e4363f11aea00481c516072fd1032825ba279bdc17e26ace96bf7"}
Jul 07 23:14:11.749913 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:11.749888    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g7tzd_c342d1b2-9987-404d-88ae-197de63d7d04/node-ca/0.log"
Jul 07 23:14:12.056106 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.056007    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:12.056283 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.056138    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:12.056283 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:12.056169    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:12.056283 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:12.056237    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 23:14:12.056283 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:12.056256    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls podName:7731486d-cda9-438c-8852-58ff55acb031 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:14.056230842 +0000 UTC m=+38.641800197 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m6fb4" (UID: "7731486d-cda9-438c-8852-58ff55acb031") : secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:12.056520 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:12.056291    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm podName:7731486d-cda9-438c-8852-58ff55acb031 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:14.056275049 +0000 UTC m=+38.641844392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m6fb4" (UID: "7731486d-cda9-438c-8852-58ff55acb031") : configmap "kube-rbac-proxy" not found
Jul 07 23:14:12.157331 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.157270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:12.157522 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:12.157432    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 23:14:12.157522 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:12.157499    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls podName:3f32a465-635f-48b6-b22a-f4e5d481db0b nodeName:}" failed. No retries permitted until 2026-07-07 23:14:14.157482616 +0000 UTC m=+38.743051958 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls") pod "dns-default-6vvnp" (UID: "3f32a465-635f-48b6-b22a-f4e5d481db0b") : secret "dns-default-metrics-tls" not found
Jul 07 23:14:12.750471 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.750432    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"]
Jul 07 23:14:12.752920 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.752900    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"
Jul 07 23:14:12.756778 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.756756    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 07 23:14:12.757156 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.757138    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 07 23:14:12.757251 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.757179    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 07 23:14:12.758097 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.758074    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-krcbt\""
Jul 07 23:14:12.758204 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.758120    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 07 23:14:12.766821 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.766745    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"]
Jul 07 23:14:12.862276 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.862241    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6a35c17c-c144-4083-b554-15f80a2261b7-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wlvx5\" (UID: \"6a35c17c-c144-4083-b554-15f80a2261b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"
Jul 07 23:14:12.862440 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.862339    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a35c17c-c144-4083-b554-15f80a2261b7-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wlvx5\" (UID: \"6a35c17c-c144-4083-b554-15f80a2261b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"
Jul 07 23:14:12.862440 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.862430    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj4bv\" (UniqueName: \"kubernetes.io/projected/6a35c17c-c144-4083-b554-15f80a2261b7-kube-api-access-bj4bv\") pod \"kube-storage-version-migrator-operator-846bf668f7-wlvx5\" (UID: \"6a35c17c-c144-4083-b554-15f80a2261b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"
Jul 07 23:14:12.963718 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.963663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj4bv\" (UniqueName: \"kubernetes.io/projected/6a35c17c-c144-4083-b554-15f80a2261b7-kube-api-access-bj4bv\") pod \"kube-storage-version-migrator-operator-846bf668f7-wlvx5\" (UID: \"6a35c17c-c144-4083-b554-15f80a2261b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"
Jul 07 23:14:12.963894 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.963738    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6a35c17c-c144-4083-b554-15f80a2261b7-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wlvx5\" (UID: \"6a35c17c-c144-4083-b554-15f80a2261b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"
Jul 07 23:14:12.963894 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.963788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a35c17c-c144-4083-b554-15f80a2261b7-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wlvx5\" (UID: \"6a35c17c-c144-4083-b554-15f80a2261b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"
Jul 07 23:14:12.964750 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.964723    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6a35c17c-c144-4083-b554-15f80a2261b7-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wlvx5\" (UID: \"6a35c17c-c144-4083-b554-15f80a2261b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"
Jul 07 23:14:12.967765 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.967737    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a35c17c-c144-4083-b554-15f80a2261b7-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wlvx5\" (UID: \"6a35c17c-c144-4083-b554-15f80a2261b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"
Jul 07 23:14:12.974321 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:12.974286    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj4bv\" (UniqueName: \"kubernetes.io/projected/6a35c17c-c144-4083-b554-15f80a2261b7-kube-api-access-bj4bv\") pod \"kube-storage-version-migrator-operator-846bf668f7-wlvx5\" (UID: \"6a35c17c-c144-4083-b554-15f80a2261b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"
Jul 07 23:14:13.064100 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:13.064008    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"
Jul 07 23:14:13.397051 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:13.397019    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5"]
Jul 07 23:14:13.400895 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:13.400738    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a35c17c_c144_4083_b554_15f80a2261b7.slice/crio-5c3b93f98bee994ba2e89a25d15b791225ca82eea456b56f55c7bf12b47ccd9c WatchSource:0}: Error finding container 5c3b93f98bee994ba2e89a25d15b791225ca82eea456b56f55c7bf12b47ccd9c: Status 404 returned error can't find the container with id 5c3b93f98bee994ba2e89a25d15b791225ca82eea456b56f55c7bf12b47ccd9c
Jul 07 23:14:13.583086 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:13.583057    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-bdm5q"]
Jul 07 23:14:13.584847 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:13.584789    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-bdm5q"
Jul 07 23:14:13.588201 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:13.588182    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ql8t7\""
Jul 07 23:14:13.599507 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:13.599476    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-bdm5q"]
Jul 07 23:14:13.771258 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:13.771223    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8h87\" (UniqueName: \"kubernetes.io/projected/1083a15d-d834-49c7-a186-f050caca1671-kube-api-access-f8h87\") pod \"network-check-source-989f8b8dc-bdm5q\" (UID: \"1083a15d-d834-49c7-a186-f050caca1671\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-bdm5q"
Jul 07 23:14:13.871962 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:13.871883    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8h87\" (UniqueName: \"kubernetes.io/projected/1083a15d-d834-49c7-a186-f050caca1671-kube-api-access-f8h87\") pod \"network-check-source-989f8b8dc-bdm5q\" (UID: \"1083a15d-d834-49c7-a186-f050caca1671\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-bdm5q"
Jul 07 23:14:13.884979 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:13.884944    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8h87\" (UniqueName: \"kubernetes.io/projected/1083a15d-d834-49c7-a186-f050caca1671-kube-api-access-f8h87\") pod \"network-check-source-989f8b8dc-bdm5q\" (UID: \"1083a15d-d834-49c7-a186-f050caca1671\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-bdm5q"
Jul 07 23:14:13.893553 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:13.893516    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-bdm5q"
Jul 07 23:14:14.074243 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:14.074208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:14.074400 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:14.074314    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:14.074446 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:14.074396    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:14.074446 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:14.074421    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 23:14:14.074530 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:14.074465    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm podName:7731486d-cda9-438c-8852-58ff55acb031 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:18.074446703 +0000 UTC m=+42.660016042 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m6fb4" (UID: "7731486d-cda9-438c-8852-58ff55acb031") : configmap "kube-rbac-proxy" not found
Jul 07 23:14:14.074530 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:14.074481    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls podName:7731486d-cda9-438c-8852-58ff55acb031 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:18.074473008 +0000 UTC m=+42.660042361 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m6fb4" (UID: "7731486d-cda9-438c-8852-58ff55acb031") : secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:14.175082 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:14.175047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:14.175266 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:14.175184    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 23:14:14.175266 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:14.175248    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls podName:3f32a465-635f-48b6-b22a-f4e5d481db0b nodeName:}" failed. No retries permitted until 2026-07-07 23:14:18.175234165 +0000 UTC m=+42.760803504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls") pod "dns-default-6vvnp" (UID: "3f32a465-635f-48b6-b22a-f4e5d481db0b") : secret "dns-default-metrics-tls" not found
Jul 07 23:14:14.220502 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:14.220467    2578 generic.go:356] "Generic (PLEG): container finished" podID="337b8d66-22e8-4cee-a345-c5dd54dca37b" containerID="27b0c0b79949ff58303614d40ff71f8a5d8281bf0b466758a4d10e3bbdaf5750" exitCode=0
Jul 07 23:14:14.220699 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:14.220580    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxn5" event={"ID":"337b8d66-22e8-4cee-a345-c5dd54dca37b","Type":"ContainerDied","Data":"27b0c0b79949ff58303614d40ff71f8a5d8281bf0b466758a4d10e3bbdaf5750"}
Jul 07 23:14:14.221908 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:14.221827    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5" event={"ID":"6a35c17c-c144-4083-b554-15f80a2261b7","Type":"ContainerStarted","Data":"5c3b93f98bee994ba2e89a25d15b791225ca82eea456b56f55c7bf12b47ccd9c"}
Jul 07 23:14:14.721783 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:14.721750    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-bdm5q"]
Jul 07 23:14:14.726381 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:14.726349    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1083a15d_d834_49c7_a186_f050caca1671.slice/crio-0c22ca8980a9676e97660472ed078ccad3b9c4124f9cd565e152556eff0761b3 WatchSource:0}: Error finding container 0c22ca8980a9676e97660472ed078ccad3b9c4124f9cd565e152556eff0761b3: Status 404 returned error can't find the container with id 0c22ca8980a9676e97660472ed078ccad3b9c4124f9cd565e152556eff0761b3
Jul 07 23:14:15.227610 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.227554    2578 generic.go:356] "Generic (PLEG): container finished" podID="337b8d66-22e8-4cee-a345-c5dd54dca37b" containerID="854a666bba303a20249d01f80368a7d2c7603bf2e6eb25f8823f76f153a5f38b" exitCode=0
Jul 07 23:14:15.227610 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.227597    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxn5" event={"ID":"337b8d66-22e8-4cee-a345-c5dd54dca37b","Type":"ContainerDied","Data":"854a666bba303a20249d01f80368a7d2c7603bf2e6eb25f8823f76f153a5f38b"}
Jul 07 23:14:15.229410 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.229383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7cbhm" event={"ID":"13d513a6-1f84-482a-985a-80339590cd66","Type":"ContainerStarted","Data":"8d49e91d6136c89e95c32dca6e5a8bfbac9da08f4f7ce6b48762fab8cd7588ff"}
Jul 07 23:14:15.230603 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.230577    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-bdm5q" event={"ID":"1083a15d-d834-49c7-a186-f050caca1671","Type":"ContainerStarted","Data":"0c22ca8980a9676e97660472ed078ccad3b9c4124f9cd565e152556eff0761b3"}
Jul 07 23:14:15.287683 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.287432    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7cbhm" podStartSLOduration=1.183992947 podStartE2EDuration="5.287413461s" totalImagesPullingTime="4.103420514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:15.285494524 +0000 UTC m=+39.871063880" watchObservedRunningTime="2026-07-07 23:14:15.287413461 +0000 UTC m=+39.872982821"
Jul 07 23:14:15.750924 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.750889    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"]
Jul 07 23:14:15.765732 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.765605    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"
Jul 07 23:14:15.765907 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.765757    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"]
Jul 07 23:14:15.769328 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.768733    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-n49f4\""
Jul 07 23:14:15.769328 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.768876    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 07 23:14:15.769328 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.769036    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 07 23:14:15.769328 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.769114    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 07 23:14:15.770058 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.770035    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 07 23:14:15.890069 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.890035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2d83076-9038-4bcb-a382-9626bf7fa985-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-vj2t7\" (UID: \"e2d83076-9038-4bcb-a382-9626bf7fa985\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"
Jul 07 23:14:15.890280 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.890206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnhsm\" (UniqueName: \"kubernetes.io/projected/e2d83076-9038-4bcb-a382-9626bf7fa985-kube-api-access-mnhsm\") pod \"service-ca-operator-5d46d9ffd4-vj2t7\" (UID: \"e2d83076-9038-4bcb-a382-9626bf7fa985\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"
Jul 07 23:14:15.890280 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.890271    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e2d83076-9038-4bcb-a382-9626bf7fa985-config\") pod \"service-ca-operator-5d46d9ffd4-vj2t7\" (UID: \"e2d83076-9038-4bcb-a382-9626bf7fa985\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"
Jul 07 23:14:15.991576 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.991528    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2d83076-9038-4bcb-a382-9626bf7fa985-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-vj2t7\" (UID: \"e2d83076-9038-4bcb-a382-9626bf7fa985\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"
Jul 07 23:14:15.991776 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.991758    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnhsm\" (UniqueName: \"kubernetes.io/projected/e2d83076-9038-4bcb-a382-9626bf7fa985-kube-api-access-mnhsm\") pod \"service-ca-operator-5d46d9ffd4-vj2t7\" (UID: \"e2d83076-9038-4bcb-a382-9626bf7fa985\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"
Jul 07 23:14:15.991842 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.991827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e2d83076-9038-4bcb-a382-9626bf7fa985-config\") pod \"service-ca-operator-5d46d9ffd4-vj2t7\" (UID: \"e2d83076-9038-4bcb-a382-9626bf7fa985\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"
Jul 07 23:14:15.992391 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.992368    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e2d83076-9038-4bcb-a382-9626bf7fa985-config\") pod \"service-ca-operator-5d46d9ffd4-vj2t7\" (UID: \"e2d83076-9038-4bcb-a382-9626bf7fa985\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"
Jul 07 23:14:15.995151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:15.995067    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2d83076-9038-4bcb-a382-9626bf7fa985-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-vj2t7\" (UID: \"e2d83076-9038-4bcb-a382-9626bf7fa985\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"
Jul 07 23:14:16.003231 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:16.003163    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnhsm\" (UniqueName: \"kubernetes.io/projected/e2d83076-9038-4bcb-a382-9626bf7fa985-kube-api-access-mnhsm\") pod \"service-ca-operator-5d46d9ffd4-vj2t7\" (UID: \"e2d83076-9038-4bcb-a382-9626bf7fa985\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"
Jul 07 23:14:16.078524 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:16.078474    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"
Jul 07 23:14:16.238311 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:16.238188    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxn5" event={"ID":"337b8d66-22e8-4cee-a345-c5dd54dca37b","Type":"ContainerStarted","Data":"3dbe6e023e3f7e3d5b4dbf51c15d80233d231ffd0fafc4ed86357f5209e43b10"}
Jul 07 23:14:16.253207 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:16.253169    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7"]
Jul 07 23:14:16.268890 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:16.268827    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5lxn5" podStartSLOduration=10.117133997 podStartE2EDuration="41.268803051s" totalImagesPullingTime="31.151669054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:16.267289461 +0000 UTC m=+40.852858829" watchObservedRunningTime="2026-07-07 23:14:16.268803051 +0000 UTC m=+40.854372415"
Jul 07 23:14:16.822069 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:16.822037    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d83076_9038_4bcb_a382_9626bf7fa985.slice/crio-4f556e5f072fae4f8344285f5c4a75cef96dfa15ab64cf2e41560218a52b07c5 WatchSource:0}: Error finding container 4f556e5f072fae4f8344285f5c4a75cef96dfa15ab64cf2e41560218a52b07c5: Status 404 returned error can't find the container with id 4f556e5f072fae4f8344285f5c4a75cef96dfa15ab64cf2e41560218a52b07c5
Jul 07 23:14:17.242913 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:17.242878    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7" event={"ID":"e2d83076-9038-4bcb-a382-9626bf7fa985","Type":"ContainerStarted","Data":"4f556e5f072fae4f8344285f5c4a75cef96dfa15ab64cf2e41560218a52b07c5"}
Jul 07 23:14:18.111533 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:18.111497    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:18.111724 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:18.111600    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:18.111724 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:18.111680    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 23:14:18.111798 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:18.111730    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm podName:7731486d-cda9-438c-8852-58ff55acb031 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:26.111717291 +0000 UTC m=+50.697286643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m6fb4" (UID: "7731486d-cda9-438c-8852-58ff55acb031") : configmap "kube-rbac-proxy" not found
Jul 07 23:14:18.111798 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:18.111680    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:18.111862 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:18.111815    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls podName:7731486d-cda9-438c-8852-58ff55acb031 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:26.111800639 +0000 UTC m=+50.697369979 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m6fb4" (UID: "7731486d-cda9-438c-8852-58ff55acb031") : secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:18.212698 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:18.212659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:18.212831 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:18.212766    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 23:14:18.212831 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:18.212819    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls podName:3f32a465-635f-48b6-b22a-f4e5d481db0b nodeName:}" failed. No retries permitted until 2026-07-07 23:14:26.21280569 +0000 UTC m=+50.798375029 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls") pod "dns-default-6vvnp" (UID: "3f32a465-635f-48b6-b22a-f4e5d481db0b") : secret "dns-default-metrics-tls" not found
Jul 07 23:14:19.250107 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:19.250068    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5" event={"ID":"6a35c17c-c144-4083-b554-15f80a2261b7","Type":"ContainerStarted","Data":"72d3cd686f119d87cc5b3c5a6c00af47fd4f88d5fa2a59cbc9201a8e84434b76"}
Jul 07 23:14:19.251800 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:19.251779    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-bdm5q" event={"ID":"1083a15d-d834-49c7-a186-f050caca1671","Type":"ContainerStarted","Data":"f7ebabee326d04836d1fcb0a1559d72ee345e6b3cc9f3078bab9431f3ecbb658"}
Jul 07 23:14:19.314236 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:19.313463    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5" podStartSLOduration=2.4281977980000002 podStartE2EDuration="7.313443099s" totalImagesPullingTime="4.885245301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:19.280444662 +0000 UTC m=+43.866014024" watchObservedRunningTime="2026-07-07 23:14:19.313443099 +0000 UTC m=+43.899012462"
Jul 07 23:14:19.314443 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:19.314406    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-bdm5q" podStartSLOduration=2.740660211 podStartE2EDuration="6.314396984s" totalImagesPullingTime="3.573736773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:19.313058486 +0000 UTC m=+43.898627848" watchObservedRunningTime="2026-07-07 23:14:19.314396984 +0000 UTC m=+43.899966348"
Jul 07 23:14:19.985751 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:19.985714    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg"]
Jul 07 23:14:20.017623 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:20.017585    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg"]
Jul 07 23:14:20.017783 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:20.017695    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg"
Jul 07 23:14:20.020497 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:20.020472    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 07 23:14:20.020497 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:20.020492    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 07 23:14:20.021459 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:20.021436    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-khz7v\""
Jul 07 23:14:20.131309 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:20.131271    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lffvf\" (UniqueName: \"kubernetes.io/projected/e2fdeee5-357d-48fb-b836-08d642f4e852-kube-api-access-lffvf\") pod \"migrator-d57d58cc9-sphqg\" (UID: \"e2fdeee5-357d-48fb-b836-08d642f4e852\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg"
Jul 07 23:14:20.231723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:20.231683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lffvf\" (UniqueName: \"kubernetes.io/projected/e2fdeee5-357d-48fb-b836-08d642f4e852-kube-api-access-lffvf\") pod \"migrator-d57d58cc9-sphqg\" (UID: \"e2fdeee5-357d-48fb-b836-08d642f4e852\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg"
Jul 07 23:14:20.241791 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:20.241728    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lffvf\" (UniqueName: \"kubernetes.io/projected/e2fdeee5-357d-48fb-b836-08d642f4e852-kube-api-access-lffvf\") pod \"migrator-d57d58cc9-sphqg\" (UID: \"e2fdeee5-357d-48fb-b836-08d642f4e852\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg"
Jul 07 23:14:20.326142 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:20.326115    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg"
Jul 07 23:14:20.463598 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:20.463549    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg"]
Jul 07 23:14:20.466442 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:20.466415    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2fdeee5_357d_48fb_b836_08d642f4e852.slice/crio-5e83ac103960fbae4dce22a5bfcffc97ef22b395aea3cc361558ee3a277283b6 WatchSource:0}: Error finding container 5e83ac103960fbae4dce22a5bfcffc97ef22b395aea3cc361558ee3a277283b6: Status 404 returned error can't find the container with id 5e83ac103960fbae4dce22a5bfcffc97ef22b395aea3cc361558ee3a277283b6
Jul 07 23:14:21.259381 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:21.259344    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7" event={"ID":"e2d83076-9038-4bcb-a382-9626bf7fa985","Type":"ContainerStarted","Data":"70c8dca9eff858c8b5e8a840ce6c3df3b030c289ffb4e2f024139828865fa80a"}
Jul 07 23:14:21.260748 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:21.260690    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg" event={"ID":"e2fdeee5-357d-48fb-b836-08d642f4e852","Type":"ContainerStarted","Data":"5e83ac103960fbae4dce22a5bfcffc97ef22b395aea3cc361558ee3a277283b6"}
Jul 07 23:14:21.282884 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:21.282813    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7" podStartSLOduration=2.827518397 podStartE2EDuration="6.282797233s" totalImagesPullingTime="3.455278836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:21.281012531 +0000 UTC m=+45.866581896" watchObservedRunningTime="2026-07-07 23:14:21.282797233 +0000 UTC m=+45.868366593"
Jul 07 23:14:23.271710 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.271678    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg" event={"ID":"e2fdeee5-357d-48fb-b836-08d642f4e852","Type":"ContainerStarted","Data":"7242cfa51a1893c22f1c5abb2e276c4d2ad1b2fa55b42d419fc3176c487410e8"}
Jul 07 23:14:23.272131 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.271739    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg" event={"ID":"e2fdeee5-357d-48fb-b836-08d642f4e852","Type":"ContainerStarted","Data":"2af70d4da07b48d9a18a60afc02f0e222b550b5d41380c7ba533938ce9d5267a"}
Jul 07 23:14:23.304293 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.304242    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sphqg" podStartSLOduration=2.595906817 podStartE2EDuration="4.304224302s" totalImagesPullingTime="1.708317485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:23.303027988 +0000 UTC m=+47.888597351" watchObservedRunningTime="2026-07-07 23:14:23.304224302 +0000 UTC m=+47.889793662"
Jul 07 23:14:23.933776 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.933735    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-2n5dl"]
Jul 07 23:14:23.956644 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.956598    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-2n5dl"
Jul 07 23:14:23.959834 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.959810    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 23:14:23.960389 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.960365    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8a728265-648c-4d50-90f0-dfb966efc948-signing-key\") pod \"service-ca-758f599c5-2n5dl\" (UID: \"8a728265-648c-4d50-90f0-dfb966efc948\") " pod="openshift-service-ca/service-ca-758f599c5-2n5dl"
Jul 07 23:14:23.960480 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.960438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8a728265-648c-4d50-90f0-dfb966efc948-signing-cabundle\") pod \"service-ca-758f599c5-2n5dl\" (UID: \"8a728265-648c-4d50-90f0-dfb966efc948\") " pod="openshift-service-ca/service-ca-758f599c5-2n5dl"
Jul 07 23:14:23.960480 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.960462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwk67\" (UniqueName: \"kubernetes.io/projected/8a728265-648c-4d50-90f0-dfb966efc948-kube-api-access-hwk67\") pod \"service-ca-758f599c5-2n5dl\" (UID: \"8a728265-648c-4d50-90f0-dfb966efc948\") " pod="openshift-service-ca/service-ca-758f599c5-2n5dl"
Jul 07 23:14:23.960480 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.960372    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 23:14:23.960689 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.960470    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 23:14:23.960749 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.960693    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-2n5dl"]
Jul 07 23:14:23.961121 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.961105    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6wd7d\""
Jul 07 23:14:23.961215 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:23.961161    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 23:14:24.061201 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:24.061166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8a728265-648c-4d50-90f0-dfb966efc948-signing-key\") pod \"service-ca-758f599c5-2n5dl\" (UID: \"8a728265-648c-4d50-90f0-dfb966efc948\") " pod="openshift-service-ca/service-ca-758f599c5-2n5dl"
Jul 07 23:14:24.061384 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:24.061233    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8a728265-648c-4d50-90f0-dfb966efc948-signing-cabundle\") pod \"service-ca-758f599c5-2n5dl\" (UID: \"8a728265-648c-4d50-90f0-dfb966efc948\") " pod="openshift-service-ca/service-ca-758f599c5-2n5dl"
Jul 07 23:14:24.061384 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:24.061259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwk67\" (UniqueName: \"kubernetes.io/projected/8a728265-648c-4d50-90f0-dfb966efc948-kube-api-access-hwk67\") pod \"service-ca-758f599c5-2n5dl\" (UID: \"8a728265-648c-4d50-90f0-dfb966efc948\") " pod="openshift-service-ca/service-ca-758f599c5-2n5dl"
Jul 07 23:14:24.062017 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:24.061997    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8a728265-648c-4d50-90f0-dfb966efc948-signing-cabundle\") pod \"service-ca-758f599c5-2n5dl\" (UID: \"8a728265-648c-4d50-90f0-dfb966efc948\") " pod="openshift-service-ca/service-ca-758f599c5-2n5dl"
Jul 07 23:14:24.063598 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:24.063578    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8a728265-648c-4d50-90f0-dfb966efc948-signing-key\") pod \"service-ca-758f599c5-2n5dl\" (UID: \"8a728265-648c-4d50-90f0-dfb966efc948\") " pod="openshift-service-ca/service-ca-758f599c5-2n5dl"
Jul 07 23:14:24.070139 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:24.070116    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwk67\" (UniqueName: \"kubernetes.io/projected/8a728265-648c-4d50-90f0-dfb966efc948-kube-api-access-hwk67\") pod \"service-ca-758f599c5-2n5dl\" (UID: \"8a728265-648c-4d50-90f0-dfb966efc948\") " pod="openshift-service-ca/service-ca-758f599c5-2n5dl"
Jul 07 23:14:24.266427 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:24.266340    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-2n5dl"
Jul 07 23:14:24.442583 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:24.442536    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-2n5dl"]
Jul 07 23:14:24.445306 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:24.445272    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a728265_648c_4d50_90f0_dfb966efc948.slice/crio-4be1f4ca1b12c5931a2ca0b1fc17e95d03bad7bb372c1c8e3bbd5e7745b98592 WatchSource:0}: Error finding container 4be1f4ca1b12c5931a2ca0b1fc17e95d03bad7bb372c1c8e3bbd5e7745b98592: Status 404 returned error can't find the container with id 4be1f4ca1b12c5931a2ca0b1fc17e95d03bad7bb372c1c8e3bbd5e7745b98592
Jul 07 23:14:25.281747 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:25.281709    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-2n5dl" event={"ID":"8a728265-648c-4d50-90f0-dfb966efc948","Type":"ContainerStarted","Data":"580093700c27eacba4139efb64b4539b2de5462c500d9ad3d98edd716b47b12c"}
Jul 07 23:14:25.281937 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:25.281763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-2n5dl" event={"ID":"8a728265-648c-4d50-90f0-dfb966efc948","Type":"ContainerStarted","Data":"4be1f4ca1b12c5931a2ca0b1fc17e95d03bad7bb372c1c8e3bbd5e7745b98592"}
Jul 07 23:14:26.181494 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:26.181455    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:26.182022 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:26.181518    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:26.182022 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:26.181689    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:26.182022 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:26.181754    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls podName:7731486d-cda9-438c-8852-58ff55acb031 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:42.181734626 +0000 UTC m=+66.767303967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m6fb4" (UID: "7731486d-cda9-438c-8852-58ff55acb031") : secret "insights-runtime-extractor-tls" not found
Jul 07 23:14:26.182192 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:26.182138    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7731486d-cda9-438c-8852-58ff55acb031-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:26.282296 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:26.282259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:26.282574 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:26.282433    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 23:14:26.282885 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:14:26.282864    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls podName:3f32a465-635f-48b6-b22a-f4e5d481db0b nodeName:}" failed. No retries permitted until 2026-07-07 23:14:42.282815787 +0000 UTC m=+66.868385127 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls") pod "dns-default-6vvnp" (UID: "3f32a465-635f-48b6-b22a-f4e5d481db0b") : secret "dns-default-metrics-tls" not found
Jul 07 23:14:37.214998 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:37.214969    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-57lng"
Jul 07 23:14:37.274855 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:37.274798    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-2n5dl" podStartSLOduration=14.274781546 podStartE2EDuration="14.274781546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:25.310488254 +0000 UTC m=+49.896057614" watchObservedRunningTime="2026-07-07 23:14:37.274781546 +0000 UTC m=+61.860350906"
Jul 07 23:14:41.602928 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.602873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:14:41.607109 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.607087    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 23:14:41.616188 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.616150    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b-metrics-certs\") pod \"network-metrics-daemon-mr5rb\" (UID: \"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b\") " pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:14:41.704229 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.704173    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:14:41.707568 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.707534    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 23:14:41.717463 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.717431    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f0cc873-6ed5-478d-b3b9-f563d8f8b773-original-pull-secret\") pod \"global-pull-secret-syncer-5kxrk\" (UID: \"5f0cc873-6ed5-478d-b3b9-f563d8f8b773\") " pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:14:41.805529 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.805492    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjvqc\" (UniqueName: \"kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc\") pod \"network-check-target-v6x6f\" (UID: \"5785bd92-cfb4-4235-9d78-47c6884049c6\") " pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:14:41.808127 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.808104    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjvqc\" (UniqueName: \"kubernetes.io/projected/5785bd92-cfb4-4235-9d78-47c6884049c6-kube-api-access-qjvqc\") pod \"network-check-target-v6x6f\" (UID: \"5785bd92-cfb4-4235-9d78-47c6884049c6\") " pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:14:41.873437 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.873413    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hxkkf\""
Jul 07 23:14:41.880050 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.880025    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mr5rb"
Jul 07 23:14:41.881614 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.881591    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nhfdw\""
Jul 07 23:14:41.886194 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.886168    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kxrk"
Jul 07 23:14:41.889908 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:41.889890    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:14:42.030701 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.030661    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mr5rb"]
Jul 07 23:14:42.036819 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:42.036793    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d9b9bc_ddd7_4bd3_b8b1_d6ead34d156b.slice/crio-e7c93ba419a95752fb6eb15a343b799141fd9cce6a9c3a3e1012677e94059a5b WatchSource:0}: Error finding container e7c93ba419a95752fb6eb15a343b799141fd9cce6a9c3a3e1012677e94059a5b: Status 404 returned error can't find the container with id e7c93ba419a95752fb6eb15a343b799141fd9cce6a9c3a3e1012677e94059a5b
Jul 07 23:14:42.209324 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.209221    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:42.211523 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.211504    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7731486d-cda9-438c-8852-58ff55acb031-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m6fb4\" (UID: \"7731486d-cda9-438c-8852-58ff55acb031\") " pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:42.252744 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.252710    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5kxrk"]
Jul 07 23:14:42.256020 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.255983    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v6x6f"]
Jul 07 23:14:42.256249 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:42.256226    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f0cc873_6ed5_478d_b3b9_f563d8f8b773.slice/crio-d310fe5082f3f8ebd43a014b71578f9c8479a3f3a319f73f6e44b38761c28bd8 WatchSource:0}: Error finding container d310fe5082f3f8ebd43a014b71578f9c8479a3f3a319f73f6e44b38761c28bd8: Status 404 returned error can't find the container with id d310fe5082f3f8ebd43a014b71578f9c8479a3f3a319f73f6e44b38761c28bd8
Jul 07 23:14:42.258730 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:42.258705    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5785bd92_cfb4_4235_9d78_47c6884049c6.slice/crio-eb2fcc983b910c64496ef2106acff8d014251440b9ad69a318e5df466ac05edc WatchSource:0}: Error finding container eb2fcc983b910c64496ef2106acff8d014251440b9ad69a318e5df466ac05edc: Status 404 returned error can't find the container with id eb2fcc983b910c64496ef2106acff8d014251440b9ad69a318e5df466ac05edc
Jul 07 23:14:42.310581 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.310524    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:42.312860 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.312832    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f32a465-635f-48b6-b22a-f4e5d481db0b-metrics-tls\") pod \"dns-default-6vvnp\" (UID: \"3f32a465-635f-48b6-b22a-f4e5d481db0b\") " pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:42.337021 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.336981    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v6x6f" event={"ID":"5785bd92-cfb4-4235-9d78-47c6884049c6","Type":"ContainerStarted","Data":"4b8568394a03f6811ae536e7769e07fda0dce97b77d7c67d6de17914bf2ce749"}
Jul 07 23:14:42.337208 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.337048    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v6x6f" event={"ID":"5785bd92-cfb4-4235-9d78-47c6884049c6","Type":"ContainerStarted","Data":"eb2fcc983b910c64496ef2106acff8d014251440b9ad69a318e5df466ac05edc"}
Jul 07 23:14:42.337208 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.337077    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:14:42.338165 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.338142    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mr5rb" event={"ID":"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b","Type":"ContainerStarted","Data":"e7c93ba419a95752fb6eb15a343b799141fd9cce6a9c3a3e1012677e94059a5b"}
Jul 07 23:14:42.339130 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.339110    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5kxrk" event={"ID":"5f0cc873-6ed5-478d-b3b9-f563d8f8b773","Type":"ContainerStarted","Data":"d310fe5082f3f8ebd43a014b71578f9c8479a3f3a319f73f6e44b38761c28bd8"}
Jul 07 23:14:42.357908 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.357858    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-v6x6f" podStartSLOduration=66.357843707 podStartE2EDuration="1m6.357843707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:13:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:42.356969247 +0000 UTC m=+66.942538607" watchObservedRunningTime="2026-07-07 23:14:42.357843707 +0000 UTC m=+66.943413088"
Jul 07 23:14:42.420019 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.419984    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wxtn5\""
Jul 07 23:14:42.427667 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.427644    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m6fb4"
Jul 07 23:14:42.514159 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.514131    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-94qrf\""
Jul 07 23:14:42.521504 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.521475    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:42.573151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.573118    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m6fb4"]
Jul 07 23:14:42.579621 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:42.579585    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7731486d_cda9_438c_8852_58ff55acb031.slice/crio-df66851cbaf55aa1faac678a63d75acc115561be29ff285c5f415aed2f70eb44 WatchSource:0}: Error finding container df66851cbaf55aa1faac678a63d75acc115561be29ff285c5f415aed2f70eb44: Status 404 returned error can't find the container with id df66851cbaf55aa1faac678a63d75acc115561be29ff285c5f415aed2f70eb44
Jul 07 23:14:42.675452 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:42.675419    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6vvnp"]
Jul 07 23:14:42.679057 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:42.679016    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f32a465_635f_48b6_b22a_f4e5d481db0b.slice/crio-5922afec73f33cb62045195dd004ec340ec026fc7e5039b67ef76fcad0853502 WatchSource:0}: Error finding container 5922afec73f33cb62045195dd004ec340ec026fc7e5039b67ef76fcad0853502: Status 404 returned error can't find the container with id 5922afec73f33cb62045195dd004ec340ec026fc7e5039b67ef76fcad0853502
Jul 07 23:14:43.348461 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:43.348336    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mr5rb" event={"ID":"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b","Type":"ContainerStarted","Data":"6a73591321528df7ecaf6479e9d208fc4ec8be8b1058cb4352169e4ab5e9c24c"}
Jul 07 23:14:43.348461 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:43.348402    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mr5rb" event={"ID":"03d9b9bc-ddd7-4bd3-b8b1-d6ead34d156b","Type":"ContainerStarted","Data":"4296137370db8413d2204b8abe3adbd82d2f9c0046c8440b2abd65d975274d84"}
Jul 07 23:14:43.351478 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:43.351018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m6fb4" event={"ID":"7731486d-cda9-438c-8852-58ff55acb031","Type":"ContainerStarted","Data":"df66851cbaf55aa1faac678a63d75acc115561be29ff285c5f415aed2f70eb44"}
Jul 07 23:14:43.353623 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:43.353426    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6vvnp" event={"ID":"3f32a465-635f-48b6-b22a-f4e5d481db0b","Type":"ContainerStarted","Data":"5922afec73f33cb62045195dd004ec340ec026fc7e5039b67ef76fcad0853502"}
Jul 07 23:14:44.380151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:44.379289    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mr5rb" podStartSLOduration=68.407732177 podStartE2EDuration="1m9.379245471s" totalImagesPullingTime="971.513294ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:44.378874742 +0000 UTC m=+68.964444118" watchObservedRunningTime="2026-07-07 23:14:44.379245471 +0000 UTC m=+68.964814832"
Jul 07 23:14:45.773510 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.773469    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"]
Jul 07 23:14:45.819670 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.819623    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"]
Jul 07 23:14:45.819863 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.819772    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:45.822946 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.822915    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 07 23:14:45.822946 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.822939    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 23:14:45.824024 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.823876    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 07 23:14:45.824024 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.823908    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 23:14:45.824024 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.823926    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 07 23:14:45.824024 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.823931    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 07 23:14:45.824290 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.823930    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 23:14:45.935718 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.935676    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-hub\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:45.935900 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.935728    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-ca\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:45.935900 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.935754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm54j\" (UniqueName: \"kubernetes.io/projected/3d721c87-12eb-4937-bdf9-3f2e63191635-kube-api-access-dm54j\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:45.935900 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.935792    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:45.935900 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.935838    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:45.935900 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:45.935877    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d721c87-12eb-4937-bdf9-3f2e63191635-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.037162 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.037069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm54j\" (UniqueName: \"kubernetes.io/projected/3d721c87-12eb-4937-bdf9-3f2e63191635-kube-api-access-dm54j\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.037162 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.037147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.037401 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.037186    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.037401 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.037224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d721c87-12eb-4937-bdf9-3f2e63191635-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.037401 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.037254    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-hub\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.037401 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.037291    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-ca\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.039122 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.039081    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d721c87-12eb-4937-bdf9-3f2e63191635-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.040486 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.040384    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.040486 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.040408    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-hub\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.040689 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.040589    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-ca\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.040689 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.040635    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d721c87-12eb-4937-bdf9-3f2e63191635-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.052937 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.052892    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm54j\" (UniqueName: \"kubernetes.io/projected/3d721c87-12eb-4937-bdf9-3f2e63191635-kube-api-access-dm54j\") pod \"cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx\" (UID: \"3d721c87-12eb-4937-bdf9-3f2e63191635\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:46.137882 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:46.137844    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"
Jul 07 23:14:49.479345 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:49.479316    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx"]
Jul 07 23:14:49.486081 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:14:49.486040    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d721c87_12eb_4937_bdf9_3f2e63191635.slice/crio-64ff52b35d9c45b7bbec7a4c4a1aa83909dca400c6508cf189104f5b158b2c58 WatchSource:0}: Error finding container 64ff52b35d9c45b7bbec7a4c4a1aa83909dca400c6508cf189104f5b158b2c58: Status 404 returned error can't find the container with id 64ff52b35d9c45b7bbec7a4c4a1aa83909dca400c6508cf189104f5b158b2c58
Jul 07 23:14:50.394239 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:50.394196    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6vvnp" event={"ID":"3f32a465-635f-48b6-b22a-f4e5d481db0b","Type":"ContainerStarted","Data":"8b506714b44db88ea1ebdf39385da44b230de2a33619685b0121417e35be7d92"}
Jul 07 23:14:50.394420 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:50.394340    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6vvnp" event={"ID":"3f32a465-635f-48b6-b22a-f4e5d481db0b","Type":"ContainerStarted","Data":"a5f445ea2fb824d95b50b280bc78299cf94960a29459a47943dc583b12f564d3"}
Jul 07 23:14:50.395834 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:50.395794    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx" event={"ID":"3d721c87-12eb-4937-bdf9-3f2e63191635","Type":"ContainerStarted","Data":"64ff52b35d9c45b7bbec7a4c4a1aa83909dca400c6508cf189104f5b158b2c58"}
Jul 07 23:14:50.398214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:50.398122    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m6fb4" event={"ID":"7731486d-cda9-438c-8852-58ff55acb031","Type":"ContainerStarted","Data":"8fad85fc15809a2383fccce906e9b8dc4b7a939c0cad19edf50f0e9d4aa6ad0c"}
Jul 07 23:14:50.398214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:50.398177    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m6fb4" event={"ID":"7731486d-cda9-438c-8852-58ff55acb031","Type":"ContainerStarted","Data":"eb1155ef33a81ff788fd77043a9316f4bc903a332afbcafbb2d4644c84035bfd"}
Jul 07 23:14:50.399943 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:50.399894    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5kxrk" event={"ID":"5f0cc873-6ed5-478d-b3b9-f563d8f8b773","Type":"ContainerStarted","Data":"cc573ac69a822e247e60dab6a50c064b80f8237e57ce9f3e87a458560ee7a911"}
Jul 07 23:14:50.417460 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:50.417399    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6vvnp" podStartSLOduration=33.802645323 podStartE2EDuration="40.417382045s" totalImagesPullingTime="6.614736722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:50.416063428 +0000 UTC m=+75.001632789" watchObservedRunningTime="2026-07-07 23:14:50.417382045 +0000 UTC m=+75.002951384"
Jul 07 23:14:50.434934 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:50.434867    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5kxrk" podStartSLOduration=67.396545123 podStartE2EDuration="1m14.434849455s" totalImagesPullingTime="7.038304332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:50.433689303 +0000 UTC m=+75.019258667" watchObservedRunningTime="2026-07-07 23:14:50.434849455 +0000 UTC m=+75.020418794"
Jul 07 23:14:51.406541 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:51.406511    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:14:53.415926 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:53.415886    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m6fb4" event={"ID":"7731486d-cda9-438c-8852-58ff55acb031","Type":"ContainerStarted","Data":"8dd28fc78c2c2773e597811f955a645db25f97860b6789e22224e16f9a4b1b95"}
Jul 07 23:14:53.438388 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:53.438331    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m6fb4" podStartSLOduration=33.12139456 podStartE2EDuration="43.438312242s" totalImagesPullingTime="10.316917682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:14:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:53.437840425 +0000 UTC m=+78.023409786" watchObservedRunningTime="2026-07-07 23:14:53.438312242 +0000 UTC m=+78.023881602"
Jul 07 23:14:56.428951 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:56.428905    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx" event={"ID":"3d721c87-12eb-4937-bdf9-3f2e63191635","Type":"ContainerStarted","Data":"8b0586f2d35010c06066ba01ecfec7a65bb556e18147b705a8bb63a662cf2f0a"}
Jul 07 23:14:56.428951 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:56.428953    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx" event={"ID":"3d721c87-12eb-4937-bdf9-3f2e63191635","Type":"ContainerStarted","Data":"059e400374ae5f5740330982a5d6ed01a6f135c43aed7e6b2c811d352f69a06e"}
Jul 07 23:14:56.429396 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:56.428966    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx" event={"ID":"3d721c87-12eb-4937-bdf9-3f2e63191635","Type":"ContainerStarted","Data":"12c46e08e53e2b7cc5388faa675a169f5edd0f117f320f0086b4bd8435227d7f"}
Jul 07 23:14:56.451195 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:14:56.451141    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f5fdbdddf-f2bjx" podStartSLOduration=5.385777194 podStartE2EDuration="11.451125554s" totalImagesPullingTime="6.06534836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:56.449040407 +0000 UTC m=+81.034609782" watchObservedRunningTime="2026-07-07 23:14:56.451125554 +0000 UTC m=+81.036694914"
Jul 07 23:15:00.737896 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.737779    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-llz4m"]
Jul 07 23:15:00.740408 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.740390    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.744400 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.744373    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 23:15:00.744547 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.744427    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 23:15:00.745441 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.745421    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 23:15:00.745441 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.745439    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 23:15:00.745611 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.745443    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9nqnm\""
Jul 07 23:15:00.745611 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.745538    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 23:15:00.745704 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.745690    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 23:15:00.854054 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.854014    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.854279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.854063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-wtmp\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.854279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.854087    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/879d5116-7b20-4d96-8406-8b4f05358e48-metrics-client-ca\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.854279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.854108    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm2hq\" (UniqueName: \"kubernetes.io/projected/879d5116-7b20-4d96-8406-8b4f05358e48-kube-api-access-fm2hq\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.854279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.854174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/879d5116-7b20-4d96-8406-8b4f05358e48-root\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.854279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.854215    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-tls\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.854279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.854242    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/879d5116-7b20-4d96-8406-8b4f05358e48-sys\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.854478 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.854276    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-accelerators-collector-config\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.854478 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.854344    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-textfile\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955240 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955403 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-wtmp\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955403 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/879d5116-7b20-4d96-8406-8b4f05358e48-metrics-client-ca\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955403 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm2hq\" (UniqueName: \"kubernetes.io/projected/879d5116-7b20-4d96-8406-8b4f05358e48-kube-api-access-fm2hq\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955403 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955363    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/879d5116-7b20-4d96-8406-8b4f05358e48-root\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955403 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955390    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-tls\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955645 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955415    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/879d5116-7b20-4d96-8406-8b4f05358e48-sys\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955645 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955440    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-accelerators-collector-config\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955645 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-textfile\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955784 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955655    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-wtmp\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955784 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955737    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/879d5116-7b20-4d96-8406-8b4f05358e48-sys\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.955929 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955906    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-textfile\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.956009 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.955990    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/879d5116-7b20-4d96-8406-8b4f05358e48-root\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.956387 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.956361    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-accelerators-collector-config\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.956576 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.956533    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/879d5116-7b20-4d96-8406-8b4f05358e48-metrics-client-ca\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.958395 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.958369    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.958873 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.958850    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/879d5116-7b20-4d96-8406-8b4f05358e48-node-exporter-tls\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:00.965340 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:00.965313    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm2hq\" (UniqueName: \"kubernetes.io/projected/879d5116-7b20-4d96-8406-8b4f05358e48-kube-api-access-fm2hq\") pod \"node-exporter-llz4m\" (UID: \"879d5116-7b20-4d96-8406-8b4f05358e48\") " pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:01.050424 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:01.050319    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-llz4m"
Jul 07 23:15:01.061756 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:15:01.061721    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod879d5116_7b20_4d96_8406_8b4f05358e48.slice/crio-034ee1b76e04e84004b1ad388aa699f3e1a6be3b1a8b05ad5961a10d59659f6d WatchSource:0}: Error finding container 034ee1b76e04e84004b1ad388aa699f3e1a6be3b1a8b05ad5961a10d59659f6d: Status 404 returned error can't find the container with id 034ee1b76e04e84004b1ad388aa699f3e1a6be3b1a8b05ad5961a10d59659f6d
Jul 07 23:15:01.412139 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:01.412105    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6vvnp"
Jul 07 23:15:01.448259 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:01.448219    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-llz4m" event={"ID":"879d5116-7b20-4d96-8406-8b4f05358e48","Type":"ContainerStarted","Data":"034ee1b76e04e84004b1ad388aa699f3e1a6be3b1a8b05ad5961a10d59659f6d"}
Jul 07 23:15:02.453119 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:02.453085    2578 generic.go:356] "Generic (PLEG): container finished" podID="879d5116-7b20-4d96-8406-8b4f05358e48" containerID="03f85dc3e1fe7e930222ec44a7d6b352540e49bd619544ac47299aac03dd30e0" exitCode=0
Jul 07 23:15:02.453599 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:02.453177    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-llz4m" event={"ID":"879d5116-7b20-4d96-8406-8b4f05358e48","Type":"ContainerDied","Data":"03f85dc3e1fe7e930222ec44a7d6b352540e49bd619544ac47299aac03dd30e0"}
Jul 07 23:15:03.458736 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:03.458704    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-llz4m" event={"ID":"879d5116-7b20-4d96-8406-8b4f05358e48","Type":"ContainerStarted","Data":"138f175caa65302d0e62cddd444f3c0cf693c528b2573ea0969267f2ecf975f5"}
Jul 07 23:15:03.459126 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:03.458754    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-llz4m" event={"ID":"879d5116-7b20-4d96-8406-8b4f05358e48","Type":"ContainerStarted","Data":"4f2e889ec0c6e1751813001cfb9057c4c43962802c45aaa54fc06a2165d46a0a"}
Jul 07 23:15:13.356202 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:13.356171    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-v6x6f"
Jul 07 23:15:13.380982 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:13.380924    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-llz4m" podStartSLOduration=12.429334643 podStartE2EDuration="13.380903766s" totalImagesPullingTime="885.849043ms" totalInitContainerRuntime="65.72008ms" isStatefulPod=true podCreationTimestamp="2026-07-07 23:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:15:03.488911179 +0000 UTC m=+88.074480540" watchObservedRunningTime="2026-07-07 23:15:13.380903766 +0000 UTC m=+97.966473129"
Jul 07 23:15:21.562039 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.562001    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58578bf6f7-dc4dd"]
Jul 07 23:15:21.576407 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.576370    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.576596 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.576524    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58578bf6f7-dc4dd"]
Jul 07 23:15:21.581182 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.581153    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 23:15:21.581182 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.581176    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 23:15:21.581387 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.581196    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 23:15:21.581387 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.581221    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m6lx8\""
Jul 07 23:15:21.581387 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.581230    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 23:15:21.581387 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.581153    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 23:15:21.581802 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.581764    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 23:15:21.581802 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.581792    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 23:15:21.586972 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.586947    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 23:15:21.714497 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.714462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-oauth-config\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.714497 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.714499    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-trusted-ca-bundle\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.714781 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.714522    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-service-ca\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.714781 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.714596    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-879kd\" (UniqueName: \"kubernetes.io/projected/0410301c-98ff-4a7a-8480-7ee43114f1c2-kube-api-access-879kd\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.714781 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.714647    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-serving-cert\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.714781 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.714666    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-oauth-serving-cert\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.714781 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.714692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-config\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.815406 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.815324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-oauth-config\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.815406 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.815367    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-trusted-ca-bundle\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.815406 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.815390    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-service-ca\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.815406 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.815410    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-879kd\" (UniqueName: \"kubernetes.io/projected/0410301c-98ff-4a7a-8480-7ee43114f1c2-kube-api-access-879kd\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.815717 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.815534    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-serving-cert\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.815717 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.815600    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-oauth-serving-cert\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.815717 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.815649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-config\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.816212 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.816185    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-service-ca\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.816381 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.816244    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-trusted-ca-bundle\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.816381 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.816314    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-config\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.816504 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.816407    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-oauth-serving-cert\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.817821 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.817803    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-oauth-config\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.818107 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.818087    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-serving-cert\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.824840 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.824805    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-879kd\" (UniqueName: \"kubernetes.io/projected/0410301c-98ff-4a7a-8480-7ee43114f1c2-kube-api-access-879kd\") pod \"console-58578bf6f7-dc4dd\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") " pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:21.886721 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:21.886682    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:22.016287 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:22.016256    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58578bf6f7-dc4dd"]
Jul 07 23:15:22.019274 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:15:22.019247    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0410301c_98ff_4a7a_8480_7ee43114f1c2.slice/crio-84fe9f5a8b62f299331054b66e581f16e4cabd349f4048f94a7d16669200172f WatchSource:0}: Error finding container 84fe9f5a8b62f299331054b66e581f16e4cabd349f4048f94a7d16669200172f: Status 404 returned error can't find the container with id 84fe9f5a8b62f299331054b66e581f16e4cabd349f4048f94a7d16669200172f
Jul 07 23:15:22.529241 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:22.529205    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58578bf6f7-dc4dd" event={"ID":"0410301c-98ff-4a7a-8480-7ee43114f1c2","Type":"ContainerStarted","Data":"84fe9f5a8b62f299331054b66e581f16e4cabd349f4048f94a7d16669200172f"}
Jul 07 23:15:25.542134 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:25.542092    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58578bf6f7-dc4dd" event={"ID":"0410301c-98ff-4a7a-8480-7ee43114f1c2","Type":"ContainerStarted","Data":"cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55"}
Jul 07 23:15:25.570908 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:25.570855    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58578bf6f7-dc4dd" podStartSLOduration=1.643863858 podStartE2EDuration="4.570836707s" totalImagesPullingTime="2.926972849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:15:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:15:25.568832714 +0000 UTC m=+110.154402075" watchObservedRunningTime="2026-07-07 23:15:25.570836707 +0000 UTC m=+110.156406069"
Jul 07 23:15:31.887842 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:31.887794    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:31.888267 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:31.887866    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:31.893613 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:31.893587    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:32.569630 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:32.569602    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:15:36.581004 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:36.580908    2578 generic.go:356] "Generic (PLEG): container finished" podID="e2d83076-9038-4bcb-a382-9626bf7fa985" containerID="70c8dca9eff858c8b5e8a840ce6c3df3b030c289ffb4e2f024139828865fa80a" exitCode=0
Jul 07 23:15:36.581004 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:36.580957    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7" event={"ID":"e2d83076-9038-4bcb-a382-9626bf7fa985","Type":"ContainerDied","Data":"70c8dca9eff858c8b5e8a840ce6c3df3b030c289ffb4e2f024139828865fa80a"}
Jul 07 23:15:36.581545 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:36.581321    2578 scope.go:117] "RemoveContainer" containerID="70c8dca9eff858c8b5e8a840ce6c3df3b030c289ffb4e2f024139828865fa80a"
Jul 07 23:15:37.586708 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:37.586677    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vj2t7" event={"ID":"e2d83076-9038-4bcb-a382-9626bf7fa985","Type":"ContainerStarted","Data":"dd4019a8fc8f4085c2838975414fbcb276859c2dd217d67f4f4512d3d102b75a"}
Jul 07 23:15:44.612738 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:44.612703    2578 generic.go:356] "Generic (PLEG): container finished" podID="6a35c17c-c144-4083-b554-15f80a2261b7" containerID="72d3cd686f119d87cc5b3c5a6c00af47fd4f88d5fa2a59cbc9201a8e84434b76" exitCode=0
Jul 07 23:15:44.613137 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:44.612755    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5" event={"ID":"6a35c17c-c144-4083-b554-15f80a2261b7","Type":"ContainerDied","Data":"72d3cd686f119d87cc5b3c5a6c00af47fd4f88d5fa2a59cbc9201a8e84434b76"}
Jul 07 23:15:44.613137 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:44.613068    2578 scope.go:117] "RemoveContainer" containerID="72d3cd686f119d87cc5b3c5a6c00af47fd4f88d5fa2a59cbc9201a8e84434b76"
Jul 07 23:15:45.618538 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:15:45.618497    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wlvx5" event={"ID":"6a35c17c-c144-4083-b554-15f80a2261b7","Type":"ContainerStarted","Data":"290494b19edeb4f3368b959bd70aa6476646c431b9892a4aa32dc23988ecdb9d"}
Jul 07 23:16:40.091907 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:16:40.091823    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58578bf6f7-dc4dd"]
Jul 07 23:17:05.112241 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.112176    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-58578bf6f7-dc4dd" podUID="0410301c-98ff-4a7a-8480-7ee43114f1c2" containerName="console" containerID="cri-o://cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55" gracePeriod=15
Jul 07 23:17:05.352114 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.352081    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:17:05.406905 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.406860    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-oauth-config\") pod \"0410301c-98ff-4a7a-8480-7ee43114f1c2\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") "
Jul 07 23:17:05.407107 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.406916    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-service-ca\") pod \"0410301c-98ff-4a7a-8480-7ee43114f1c2\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") "
Jul 07 23:17:05.407107 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.406938    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-config\") pod \"0410301c-98ff-4a7a-8480-7ee43114f1c2\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") "
Jul 07 23:17:05.407107 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.406967    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-879kd\" (UniqueName: \"kubernetes.io/projected/0410301c-98ff-4a7a-8480-7ee43114f1c2-kube-api-access-879kd\") pod \"0410301c-98ff-4a7a-8480-7ee43114f1c2\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") "
Jul 07 23:17:05.407107 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.406982    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-serving-cert\") pod \"0410301c-98ff-4a7a-8480-7ee43114f1c2\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") "
Jul 07 23:17:05.407107 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.407010    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-oauth-serving-cert\") pod \"0410301c-98ff-4a7a-8480-7ee43114f1c2\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") "
Jul 07 23:17:05.407107 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.407029    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-trusted-ca-bundle\") pod \"0410301c-98ff-4a7a-8480-7ee43114f1c2\" (UID: \"0410301c-98ff-4a7a-8480-7ee43114f1c2\") "
Jul 07 23:17:05.407467 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.407433    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-service-ca" (OuterVolumeSpecName: "service-ca") pod "0410301c-98ff-4a7a-8480-7ee43114f1c2" (UID: "0410301c-98ff-4a7a-8480-7ee43114f1c2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:17:05.407546 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.407508    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-config" (OuterVolumeSpecName: "console-config") pod "0410301c-98ff-4a7a-8480-7ee43114f1c2" (UID: "0410301c-98ff-4a7a-8480-7ee43114f1c2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:17:05.407613 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.407538    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0410301c-98ff-4a7a-8480-7ee43114f1c2" (UID: "0410301c-98ff-4a7a-8480-7ee43114f1c2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:17:05.407613 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.407598    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0410301c-98ff-4a7a-8480-7ee43114f1c2" (UID: "0410301c-98ff-4a7a-8480-7ee43114f1c2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:17:05.409302 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.409273    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0410301c-98ff-4a7a-8480-7ee43114f1c2" (UID: "0410301c-98ff-4a7a-8480-7ee43114f1c2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:17:05.409302 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.409289    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0410301c-98ff-4a7a-8480-7ee43114f1c2-kube-api-access-879kd" (OuterVolumeSpecName: "kube-api-access-879kd") pod "0410301c-98ff-4a7a-8480-7ee43114f1c2" (UID: "0410301c-98ff-4a7a-8480-7ee43114f1c2"). InnerVolumeSpecName "kube-api-access-879kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:17:05.409436 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.409285    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0410301c-98ff-4a7a-8480-7ee43114f1c2" (UID: "0410301c-98ff-4a7a-8480-7ee43114f1c2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:17:05.507581 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.507504    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-oauth-config\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:17:05.507581 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.507553    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:17:05.507581 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.507587    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-config\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:17:05.507581 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.507595    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-879kd\" (UniqueName: \"kubernetes.io/projected/0410301c-98ff-4a7a-8480-7ee43114f1c2-kube-api-access-879kd\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:17:05.507856 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.507604    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0410301c-98ff-4a7a-8480-7ee43114f1c2-console-serving-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:17:05.507856 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.507613    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-oauth-serving-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:17:05.507856 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.507621    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0410301c-98ff-4a7a-8480-7ee43114f1c2-trusted-ca-bundle\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:17:05.896737 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.896703    2578 generic.go:356] "Generic (PLEG): container finished" podID="0410301c-98ff-4a7a-8480-7ee43114f1c2" containerID="cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55" exitCode=0
Jul 07 23:17:05.896935 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.896757    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58578bf6f7-dc4dd" event={"ID":"0410301c-98ff-4a7a-8480-7ee43114f1c2","Type":"ContainerDied","Data":"cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55"}
Jul 07 23:17:05.896935 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.896761    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58578bf6f7-dc4dd"
Jul 07 23:17:05.896935 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.896852    2578 scope.go:117] "RemoveContainer" containerID="cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55"
Jul 07 23:17:05.896935 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.896838    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58578bf6f7-dc4dd" event={"ID":"0410301c-98ff-4a7a-8480-7ee43114f1c2","Type":"ContainerDied","Data":"84fe9f5a8b62f299331054b66e581f16e4cabd349f4048f94a7d16669200172f"}
Jul 07 23:17:05.905631 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.905611    2578 scope.go:117] "RemoveContainer" containerID="cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55"
Jul 07 23:17:05.905921 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:17:05.905905    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55\": container with ID starting with cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55 not found: ID does not exist" containerID="cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55"
Jul 07 23:17:05.905973 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.905929    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55"} err="failed to get container status \"cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55\": rpc error: code = NotFound desc = could not find container \"cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55\": container with ID starting with cc81cdd6fd06eb4eac8540f21c0fa8b375cf00c2baef69a7a8c34bd399910e55 not found: ID does not exist"
Jul 07 23:17:05.920722 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.920688    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58578bf6f7-dc4dd"]
Jul 07 23:17:05.928902 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.928872    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58578bf6f7-dc4dd"]
Jul 07 23:17:05.960856 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:17:05.960827    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0410301c-98ff-4a7a-8480-7ee43114f1c2" path="/var/lib/kubelet/pods/0410301c-98ff-4a7a-8480-7ee43114f1c2/volumes"
Jul 07 23:18:35.771999 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:18:35.771969    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 23:19:07.631063 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.631024    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kx42l"]
Jul 07 23:19:07.633494 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.631358    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0410301c-98ff-4a7a-8480-7ee43114f1c2" containerName="console"
Jul 07 23:19:07.633494 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.631375    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0410301c-98ff-4a7a-8480-7ee43114f1c2" containerName="console"
Jul 07 23:19:07.633494 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.631456    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0410301c-98ff-4a7a-8480-7ee43114f1c2" containerName="console"
Jul 07 23:19:07.634414 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.634396    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kx42l"
Jul 07 23:19:07.637352 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.637330    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 07 23:19:07.638304 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.638285    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-27jpx\""
Jul 07 23:19:07.638408 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.638334    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 07 23:19:07.642721 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.642698    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kx42l"]
Jul 07 23:19:07.776145 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.776107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b75s\" (UniqueName: \"kubernetes.io/projected/f7e864e5-eeec-41c6-bfcc-78e195098939-kube-api-access-6b75s\") pod \"cert-manager-cainjector-559cdd8c87-kx42l\" (UID: \"f7e864e5-eeec-41c6-bfcc-78e195098939\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kx42l"
Jul 07 23:19:07.776323 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.776155    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7e864e5-eeec-41c6-bfcc-78e195098939-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kx42l\" (UID: \"f7e864e5-eeec-41c6-bfcc-78e195098939\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kx42l"
Jul 07 23:19:07.876854 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.876822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7e864e5-eeec-41c6-bfcc-78e195098939-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kx42l\" (UID: \"f7e864e5-eeec-41c6-bfcc-78e195098939\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kx42l"
Jul 07 23:19:07.877027 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.876896    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b75s\" (UniqueName: \"kubernetes.io/projected/f7e864e5-eeec-41c6-bfcc-78e195098939-kube-api-access-6b75s\") pod \"cert-manager-cainjector-559cdd8c87-kx42l\" (UID: \"f7e864e5-eeec-41c6-bfcc-78e195098939\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kx42l"
Jul 07 23:19:07.886974 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.886896    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7e864e5-eeec-41c6-bfcc-78e195098939-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kx42l\" (UID: \"f7e864e5-eeec-41c6-bfcc-78e195098939\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kx42l"
Jul 07 23:19:07.886974 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.886919    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b75s\" (UniqueName: \"kubernetes.io/projected/f7e864e5-eeec-41c6-bfcc-78e195098939-kube-api-access-6b75s\") pod \"cert-manager-cainjector-559cdd8c87-kx42l\" (UID: \"f7e864e5-eeec-41c6-bfcc-78e195098939\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kx42l"
Jul 07 23:19:07.944003 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:07.943950    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kx42l"
Jul 07 23:19:08.082039 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:08.082006    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kx42l"]
Jul 07 23:19:08.085294 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:19:08.085262    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7e864e5_eeec_41c6_bfcc_78e195098939.slice/crio-701bdd729ce040091c438dc457221a774d7462a86a2e39a8baf20d4ce2c7469d WatchSource:0}: Error finding container 701bdd729ce040091c438dc457221a774d7462a86a2e39a8baf20d4ce2c7469d: Status 404 returned error can't find the container with id 701bdd729ce040091c438dc457221a774d7462a86a2e39a8baf20d4ce2c7469d
Jul 07 23:19:08.087136 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:08.087118    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 23:19:08.316257 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:08.316169    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kx42l" event={"ID":"f7e864e5-eeec-41c6-bfcc-78e195098939","Type":"ContainerStarted","Data":"701bdd729ce040091c438dc457221a774d7462a86a2e39a8baf20d4ce2c7469d"}
Jul 07 23:19:11.329488 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:11.329384    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kx42l" event={"ID":"f7e864e5-eeec-41c6-bfcc-78e195098939","Type":"ContainerStarted","Data":"79d6aff0a8e0fae850a067779949bf8db1bbf66c47ddb85dd81f16f63882ada8"}
Jul 07 23:19:11.347810 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:11.347753    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kx42l" podStartSLOduration=1.46251073 podStartE2EDuration="4.347732883s" totalImagesPullingTime="2.885222153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:19:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:19:11.346826737 +0000 UTC m=+335.932396098" watchObservedRunningTime="2026-07-07 23:19:11.347732883 +0000 UTC m=+335.933302244"
Jul 07 23:19:15.116417 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.116329    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w8tq8"]
Jul 07 23:19:15.119407 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.119389    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-w8tq8"
Jul 07 23:19:15.121914 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.121895    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-j2pkv\""
Jul 07 23:19:15.133272 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.133232    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w8tq8"]
Jul 07 23:19:15.233220 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.233174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z24ht\" (UniqueName: \"kubernetes.io/projected/67a047de-a2d0-48c6-81c8-c9807214f466-kube-api-access-z24ht\") pod \"cert-manager-55b4d954d5-w8tq8\" (UID: \"67a047de-a2d0-48c6-81c8-c9807214f466\") " pod="cert-manager/cert-manager-55b4d954d5-w8tq8"
Jul 07 23:19:15.233408 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.233231    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/67a047de-a2d0-48c6-81c8-c9807214f466-bound-sa-token\") pod \"cert-manager-55b4d954d5-w8tq8\" (UID: \"67a047de-a2d0-48c6-81c8-c9807214f466\") " pod="cert-manager/cert-manager-55b4d954d5-w8tq8"
Jul 07 23:19:15.333671 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.333634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z24ht\" (UniqueName: \"kubernetes.io/projected/67a047de-a2d0-48c6-81c8-c9807214f466-kube-api-access-z24ht\") pod \"cert-manager-55b4d954d5-w8tq8\" (UID: \"67a047de-a2d0-48c6-81c8-c9807214f466\") " pod="cert-manager/cert-manager-55b4d954d5-w8tq8"
Jul 07 23:19:15.333810 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.333679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/67a047de-a2d0-48c6-81c8-c9807214f466-bound-sa-token\") pod \"cert-manager-55b4d954d5-w8tq8\" (UID: \"67a047de-a2d0-48c6-81c8-c9807214f466\") " pod="cert-manager/cert-manager-55b4d954d5-w8tq8"
Jul 07 23:19:15.343180 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.343148    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/67a047de-a2d0-48c6-81c8-c9807214f466-bound-sa-token\") pod \"cert-manager-55b4d954d5-w8tq8\" (UID: \"67a047de-a2d0-48c6-81c8-c9807214f466\") " pod="cert-manager/cert-manager-55b4d954d5-w8tq8"
Jul 07 23:19:15.343315 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.343284    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z24ht\" (UniqueName: \"kubernetes.io/projected/67a047de-a2d0-48c6-81c8-c9807214f466-kube-api-access-z24ht\") pod \"cert-manager-55b4d954d5-w8tq8\" (UID: \"67a047de-a2d0-48c6-81c8-c9807214f466\") " pod="cert-manager/cert-manager-55b4d954d5-w8tq8"
Jul 07 23:19:15.428349 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.428310    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-w8tq8"
Jul 07 23:19:15.559404 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:15.559368    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w8tq8"]
Jul 07 23:19:15.562173 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:19:15.562147    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67a047de_a2d0_48c6_81c8_c9807214f466.slice/crio-2234e5a9c48301f351c4622a49b16814060ddd32e62a2c39f2372da6b7f92a9a WatchSource:0}: Error finding container 2234e5a9c48301f351c4622a49b16814060ddd32e62a2c39f2372da6b7f92a9a: Status 404 returned error can't find the container with id 2234e5a9c48301f351c4622a49b16814060ddd32e62a2c39f2372da6b7f92a9a
Jul 07 23:19:16.348851 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:16.348820    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-w8tq8" event={"ID":"67a047de-a2d0-48c6-81c8-c9807214f466","Type":"ContainerStarted","Data":"6b17a0ff041b02102dfd02f57b45d8ce3b173a77bfbd1dd8949f769b381f1120"}
Jul 07 23:19:16.349329 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:16.348859    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-w8tq8" event={"ID":"67a047de-a2d0-48c6-81c8-c9807214f466","Type":"ContainerStarted","Data":"2234e5a9c48301f351c4622a49b16814060ddd32e62a2c39f2372da6b7f92a9a"}
Jul 07 23:19:16.381527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:16.381472    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-w8tq8" podStartSLOduration=1.381456572 podStartE2EDuration="1.381456572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:19:16.381310807 +0000 UTC m=+340.966880169" watchObservedRunningTime="2026-07-07 23:19:16.381456572 +0000 UTC m=+340.967025933"
Jul 07 23:19:44.305771 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.305685    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"]
Jul 07 23:19:44.311977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.311953    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.316988 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.316952    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 23:19:44.317439 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.317418    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 23:19:44.317728 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.317706    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 23:19:44.317833 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.317730    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 23:19:44.317833 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.317759    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-z2ktg\""
Jul 07 23:19:44.336113 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.336066    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzqxn\" (UniqueName: \"kubernetes.io/projected/4d880249-e1c5-4c06-95b0-3fc875533f36-kube-api-access-tzqxn\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.336113 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.336119    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4d880249-e1c5-4c06-95b0-3fc875533f36-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.336380 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.336240    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4d880249-e1c5-4c06-95b0-3fc875533f36-manifests\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.336380 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.336274    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d880249-e1c5-4c06-95b0-3fc875533f36-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.336380 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.336338    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d880249-e1c5-4c06-95b0-3fc875533f36-tmp\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.368837 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.368801    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"]
Jul 07 23:19:44.437666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.437626    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzqxn\" (UniqueName: \"kubernetes.io/projected/4d880249-e1c5-4c06-95b0-3fc875533f36-kube-api-access-tzqxn\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.437666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.437666    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4d880249-e1c5-4c06-95b0-3fc875533f36-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.437926 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.437690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4d880249-e1c5-4c06-95b0-3fc875533f36-manifests\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.437926 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.437704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d880249-e1c5-4c06-95b0-3fc875533f36-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.437926 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.437728    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d880249-e1c5-4c06-95b0-3fc875533f36-tmp\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.438095 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.438051    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4d880249-e1c5-4c06-95b0-3fc875533f36-manifests\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.440023 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.439999    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d880249-e1c5-4c06-95b0-3fc875533f36-tmp\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.440263 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.440243    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d880249-e1c5-4c06-95b0-3fc875533f36-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.440337 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.440314    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4d880249-e1c5-4c06-95b0-3fc875533f36-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.468405 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.468369    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzqxn\" (UniqueName: \"kubernetes.io/projected/4d880249-e1c5-4c06-95b0-3fc875533f36-kube-api-access-tzqxn\") pod \"opendatahub-operator-controller-manager-7c9859966d-f6dzb\" (UID: \"4d880249-e1c5-4c06-95b0-3fc875533f36\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.623039 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.622944    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:19:44.813376 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:44.813351    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"]
Jul 07 23:19:44.815256 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:19:44.815230    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d880249_e1c5_4c06_95b0_3fc875533f36.slice/crio-b44b9f533413141430e39db6ff5225f7618d63e242cf63fd917b6c37eedaee75 WatchSource:0}: Error finding container b44b9f533413141430e39db6ff5225f7618d63e242cf63fd917b6c37eedaee75: Status 404 returned error can't find the container with id b44b9f533413141430e39db6ff5225f7618d63e242cf63fd917b6c37eedaee75
Jul 07 23:19:45.456966 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:45.456888    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb" event={"ID":"4d880249-e1c5-4c06-95b0-3fc875533f36","Type":"ContainerStarted","Data":"b44b9f533413141430e39db6ff5225f7618d63e242cf63fd917b6c37eedaee75"}
Jul 07 23:19:48.472913 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:48.472874    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb" event={"ID":"4d880249-e1c5-4c06-95b0-3fc875533f36","Type":"ContainerStarted","Data":"2710cad32aec5b6bdcae93ed123ccc3053efc156640b12c3080d017889b1b7d3"}
Jul 07 23:19:49.478144 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:49.478113    2578 generic.go:356] "Generic (PLEG): container finished" podID="4d880249-e1c5-4c06-95b0-3fc875533f36" containerID="2710cad32aec5b6bdcae93ed123ccc3053efc156640b12c3080d017889b1b7d3" exitCode=0
Jul 07 23:19:49.478524 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:49.478176    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb" event={"ID":"4d880249-e1c5-4c06-95b0-3fc875533f36","Type":"ContainerDied","Data":"2710cad32aec5b6bdcae93ed123ccc3053efc156640b12c3080d017889b1b7d3"}
Jul 07 23:19:50.483991 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:50.483954    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb" event={"ID":"4d880249-e1c5-4c06-95b0-3fc875533f36","Type":"ContainerStarted","Data":"569fef26d87333052aaca9385749905d8e6b09c266ac3c65d492022f2a98a72e"}
Jul 07 23:19:50.484403 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:19:50.484088    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:20:01.490399 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:01.490369    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb"
Jul 07 23:20:01.517228 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:01.517165    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-f6dzb" podStartSLOduration=13.173878751 podStartE2EDuration="17.517145804s" totalImagesPullingTime="3.332194367s" totalInitContainerRuntime="1.011072686s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:19:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:19:50.547120949 +0000 UTC m=+375.132690310" watchObservedRunningTime="2026-07-07 23:20:01.517145804 +0000 UTC m=+386.102715164"
Jul 07 23:20:04.380951 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.380915    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"]
Jul 07 23:20:04.389060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.389031    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"
Jul 07 23:20:04.402430 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.402401    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"]
Jul 07 23:20:04.403206 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.403189    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 23:20:04.403636 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.403616    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 23:20:04.403764 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.403693    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xf9mp\""
Jul 07 23:20:04.404311 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.404297    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 23:20:04.404403 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.404365    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 23:20:04.490505 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.490468    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5abf2439-622a-4389-b746-d87f33ea9c7d-tmp\") pod \"kube-auth-proxy-7b7668c465-5rrp6\" (UID: \"5abf2439-622a-4389-b746-d87f33ea9c7d\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"
Jul 07 23:20:04.490505 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.490508    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhsjk\" (UniqueName: \"kubernetes.io/projected/5abf2439-622a-4389-b746-d87f33ea9c7d-kube-api-access-bhsjk\") pod \"kube-auth-proxy-7b7668c465-5rrp6\" (UID: \"5abf2439-622a-4389-b746-d87f33ea9c7d\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"
Jul 07 23:20:04.490769 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.490544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5abf2439-622a-4389-b746-d87f33ea9c7d-tls-certs\") pod \"kube-auth-proxy-7b7668c465-5rrp6\" (UID: \"5abf2439-622a-4389-b746-d87f33ea9c7d\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"
Jul 07 23:20:04.591689 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.591655    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5abf2439-622a-4389-b746-d87f33ea9c7d-tmp\") pod \"kube-auth-proxy-7b7668c465-5rrp6\" (UID: \"5abf2439-622a-4389-b746-d87f33ea9c7d\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"
Jul 07 23:20:04.591689 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.591698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhsjk\" (UniqueName: \"kubernetes.io/projected/5abf2439-622a-4389-b746-d87f33ea9c7d-kube-api-access-bhsjk\") pod \"kube-auth-proxy-7b7668c465-5rrp6\" (UID: \"5abf2439-622a-4389-b746-d87f33ea9c7d\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"
Jul 07 23:20:04.591900 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.591731    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5abf2439-622a-4389-b746-d87f33ea9c7d-tls-certs\") pod \"kube-auth-proxy-7b7668c465-5rrp6\" (UID: \"5abf2439-622a-4389-b746-d87f33ea9c7d\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"
Jul 07 23:20:04.593949 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.593914    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5abf2439-622a-4389-b746-d87f33ea9c7d-tmp\") pod \"kube-auth-proxy-7b7668c465-5rrp6\" (UID: \"5abf2439-622a-4389-b746-d87f33ea9c7d\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"
Jul 07 23:20:04.594165 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.594138    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5abf2439-622a-4389-b746-d87f33ea9c7d-tls-certs\") pod \"kube-auth-proxy-7b7668c465-5rrp6\" (UID: \"5abf2439-622a-4389-b746-d87f33ea9c7d\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"
Jul 07 23:20:04.603159 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.603128    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhsjk\" (UniqueName: \"kubernetes.io/projected/5abf2439-622a-4389-b746-d87f33ea9c7d-kube-api-access-bhsjk\") pod \"kube-auth-proxy-7b7668c465-5rrp6\" (UID: \"5abf2439-622a-4389-b746-d87f33ea9c7d\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"
Jul 07 23:20:04.698087 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.698003    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"
Jul 07 23:20:04.714173 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.714135    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jk5xl"]
Jul 07 23:20:04.717590 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.717550    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl"
Jul 07 23:20:04.720547 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.720514    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-8b2nc\""
Jul 07 23:20:04.720802 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.720784    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 07 23:20:04.730159 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.730121    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jk5xl"]
Jul 07 23:20:04.793850 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.793819    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj4lb\" (UniqueName: \"kubernetes.io/projected/312d941a-d76a-4cb8-aa34-7b4d5271bd20-kube-api-access-vj4lb\") pod \"odh-model-controller-7998c67dc4-jk5xl\" (UID: \"312d941a-d76a-4cb8-aa34-7b4d5271bd20\") " pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl"
Jul 07 23:20:04.793976 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.793870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/312d941a-d76a-4cb8-aa34-7b4d5271bd20-cert\") pod \"odh-model-controller-7998c67dc4-jk5xl\" (UID: \"312d941a-d76a-4cb8-aa34-7b4d5271bd20\") " pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl"
Jul 07 23:20:04.860128 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.860103    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6"]
Jul 07 23:20:04.862887 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:20:04.862848    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5abf2439_622a_4389_b746_d87f33ea9c7d.slice/crio-925ba4761fd914392d444fe5a4f806b3a07c82c51eb247cdd526808c6d8932af WatchSource:0}: Error finding container 925ba4761fd914392d444fe5a4f806b3a07c82c51eb247cdd526808c6d8932af: Status 404 returned error can't find the container with id 925ba4761fd914392d444fe5a4f806b3a07c82c51eb247cdd526808c6d8932af
Jul 07 23:20:04.894729 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.894687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj4lb\" (UniqueName: \"kubernetes.io/projected/312d941a-d76a-4cb8-aa34-7b4d5271bd20-kube-api-access-vj4lb\") pod \"odh-model-controller-7998c67dc4-jk5xl\" (UID: \"312d941a-d76a-4cb8-aa34-7b4d5271bd20\") " pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl"
Jul 07 23:20:04.894729 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.894727    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/312d941a-d76a-4cb8-aa34-7b4d5271bd20-cert\") pod \"odh-model-controller-7998c67dc4-jk5xl\" (UID: \"312d941a-d76a-4cb8-aa34-7b4d5271bd20\") " pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl"
Jul 07 23:20:04.894977 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:20:04.894839    2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 07 23:20:04.894977 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:20:04.894902    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/312d941a-d76a-4cb8-aa34-7b4d5271bd20-cert podName:312d941a-d76a-4cb8-aa34-7b4d5271bd20 nodeName:}" failed. No retries permitted until 2026-07-07 23:20:05.394884493 +0000 UTC m=+389.980453832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/312d941a-d76a-4cb8-aa34-7b4d5271bd20-cert") pod "odh-model-controller-7998c67dc4-jk5xl" (UID: "312d941a-d76a-4cb8-aa34-7b4d5271bd20") : secret "odh-model-controller-webhook-cert" not found
Jul 07 23:20:04.904253 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:04.904224    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj4lb\" (UniqueName: \"kubernetes.io/projected/312d941a-d76a-4cb8-aa34-7b4d5271bd20-kube-api-access-vj4lb\") pod \"odh-model-controller-7998c67dc4-jk5xl\" (UID: \"312d941a-d76a-4cb8-aa34-7b4d5271bd20\") " pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl"
Jul 07 23:20:05.399017 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.398978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/312d941a-d76a-4cb8-aa34-7b4d5271bd20-cert\") pod \"odh-model-controller-7998c67dc4-jk5xl\" (UID: \"312d941a-d76a-4cb8-aa34-7b4d5271bd20\") " pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl"
Jul 07 23:20:05.401383 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.401362    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/312d941a-d76a-4cb8-aa34-7b4d5271bd20-cert\") pod \"odh-model-controller-7998c67dc4-jk5xl\" (UID: \"312d941a-d76a-4cb8-aa34-7b4d5271bd20\") " pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl"
Jul 07 23:20:05.539597 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.539544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6" event={"ID":"5abf2439-622a-4389-b746-d87f33ea9c7d","Type":"ContainerStarted","Data":"925ba4761fd914392d444fe5a4f806b3a07c82c51eb247cdd526808c6d8932af"}
Jul 07 23:20:05.631898 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.631865    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl"
Jul 07 23:20:05.718460 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.718415    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vlkvz"]
Jul 07 23:20:05.721888 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.721808    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz"
Jul 07 23:20:05.725108 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.725080    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-xl8xx\""
Jul 07 23:20:05.726590 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.726550    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 07 23:20:05.735044 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.734634    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vlkvz"]
Jul 07 23:20:05.800624 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.800588    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jk5xl"]
Jul 07 23:20:05.802444 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.802417    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/719d2eb5-c1d4-45bf-adc5-dd16f0328209-tls-certs\") pod \"model-serving-api-54d89bd5fc-vlkvz\" (UID: \"719d2eb5-c1d4-45bf-adc5-dd16f0328209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz"
Jul 07 23:20:05.802544 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.802462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv95d\" (UniqueName: \"kubernetes.io/projected/719d2eb5-c1d4-45bf-adc5-dd16f0328209-kube-api-access-tv95d\") pod \"model-serving-api-54d89bd5fc-vlkvz\" (UID: \"719d2eb5-c1d4-45bf-adc5-dd16f0328209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz"
Jul 07 23:20:05.804057 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:20:05.804033    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312d941a_d76a_4cb8_aa34_7b4d5271bd20.slice/crio-b6e9f7952c1bc998f653b85db7a232ad2e411e04296fd09e431b6dbd4a898fbb WatchSource:0}: Error finding container b6e9f7952c1bc998f653b85db7a232ad2e411e04296fd09e431b6dbd4a898fbb: Status 404 returned error can't find the container with id b6e9f7952c1bc998f653b85db7a232ad2e411e04296fd09e431b6dbd4a898fbb
Jul 07 23:20:05.903487 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.903457    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/719d2eb5-c1d4-45bf-adc5-dd16f0328209-tls-certs\") pod \"model-serving-api-54d89bd5fc-vlkvz\" (UID: \"719d2eb5-c1d4-45bf-adc5-dd16f0328209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz"
Jul 07 23:20:05.903708 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.903498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv95d\" (UniqueName: \"kubernetes.io/projected/719d2eb5-c1d4-45bf-adc5-dd16f0328209-kube-api-access-tv95d\") pod \"model-serving-api-54d89bd5fc-vlkvz\" (UID: \"719d2eb5-c1d4-45bf-adc5-dd16f0328209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz"
Jul 07 23:20:05.906618 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.906590    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/719d2eb5-c1d4-45bf-adc5-dd16f0328209-tls-certs\") pod \"model-serving-api-54d89bd5fc-vlkvz\" (UID: \"719d2eb5-c1d4-45bf-adc5-dd16f0328209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz"
Jul 07 23:20:05.921052 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.921025    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv95d\" (UniqueName: \"kubernetes.io/projected/719d2eb5-c1d4-45bf-adc5-dd16f0328209-kube-api-access-tv95d\") pod \"model-serving-api-54d89bd5fc-vlkvz\" (UID: \"719d2eb5-c1d4-45bf-adc5-dd16f0328209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz"
Jul 07 23:20:05.994139 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.994050    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"]
Jul 07 23:20:05.997752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:05.997712    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.001050 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.001026    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 23:20:06.001198 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.001060    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 07 23:20:06.001198 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.001098    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bn98l\""
Jul 07 23:20:06.001198 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.001026    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 07 23:20:06.001664 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.001642    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 07 23:20:06.017173 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.017138    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"]
Jul 07 23:20:06.035733 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.035703    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz"
Jul 07 23:20:06.105373 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.105338    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tqhh\" (UniqueName: \"kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-kube-api-access-5tqhh\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.105373 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.105377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-token\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.105608 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.105395    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.105608 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.105411    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-cacerts\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.105608 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.105463    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.105608 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.105492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/757c5ad7-3cea-435f-be0e-06e9e9c91a30-local-certs\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.105608 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.105590    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.178592 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.178491    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vlkvz"]
Jul 07 23:20:06.181158 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:20:06.181125    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod719d2eb5_c1d4_45bf_adc5_dd16f0328209.slice/crio-249fb554763ab0dfabc70061e92e102b29774ccf788cc0abe6c550702a51bbe9 WatchSource:0}: Error finding container 249fb554763ab0dfabc70061e92e102b29774ccf788cc0abe6c550702a51bbe9: Status 404 returned error can't find the container with id 249fb554763ab0dfabc70061e92e102b29774ccf788cc0abe6c550702a51bbe9
Jul 07 23:20:06.206794 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.206755    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.206985 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.206814    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tqhh\" (UniqueName: \"kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-kube-api-access-5tqhh\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.206985 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.206855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-token\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.206985 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.206883    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.206985 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.206934    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-cacerts\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.206985 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.206968    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.207247 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.207024    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/757c5ad7-3cea-435f-be0e-06e9e9c91a30-local-certs\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.207768 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.207694    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.209680 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.209654    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/757c5ad7-3cea-435f-be0e-06e9e9c91a30-local-certs\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.209909 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.209888    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.210013 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.209923    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-cacerts\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.210013 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.209999    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.216225 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.216195    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tqhh\" (UniqueName: \"kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-kube-api-access-5tqhh\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.216370 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.216323    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-token\") pod \"istiod-openshift-gateway-7574459656-7sfx6\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.309119 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.309019    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:06.462256 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.462225    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"]
Jul 07 23:20:06.465937 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:20:06.465904    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod757c5ad7_3cea_435f_be0e_06e9e9c91a30.slice/crio-bb5a01f8e383e4b409bc39b42edb1eb5811491cd939c49d51bde3b6d14b96283 WatchSource:0}: Error finding container bb5a01f8e383e4b409bc39b42edb1eb5811491cd939c49d51bde3b6d14b96283: Status 404 returned error can't find the container with id bb5a01f8e383e4b409bc39b42edb1eb5811491cd939c49d51bde3b6d14b96283
Jul 07 23:20:06.545511 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.545473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz" event={"ID":"719d2eb5-c1d4-45bf-adc5-dd16f0328209","Type":"ContainerStarted","Data":"249fb554763ab0dfabc70061e92e102b29774ccf788cc0abe6c550702a51bbe9"}
Jul 07 23:20:06.546922 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.546886    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6" event={"ID":"757c5ad7-3cea-435f-be0e-06e9e9c91a30","Type":"ContainerStarted","Data":"bb5a01f8e383e4b409bc39b42edb1eb5811491cd939c49d51bde3b6d14b96283"}
Jul 07 23:20:06.548104 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:06.548080    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl" event={"ID":"312d941a-d76a-4cb8-aa34-7b4d5271bd20","Type":"ContainerStarted","Data":"b6e9f7952c1bc998f653b85db7a232ad2e411e04296fd09e431b6dbd4a898fbb"}
Jul 07 23:20:10.568572 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:10.568522    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz" event={"ID":"719d2eb5-c1d4-45bf-adc5-dd16f0328209","Type":"ContainerStarted","Data":"4e13bb5395bc656aac24009992493e3f2e6d1a1f9dcd2d3cea2f4c02161752e8"}
Jul 07 23:20:10.569050 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:10.568642    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz"
Jul 07 23:20:10.570104 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:10.570081    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6" event={"ID":"5abf2439-622a-4389-b746-d87f33ea9c7d","Type":"ContainerStarted","Data":"f3ceb24b1b9c8bba8817b02b98ab84be9b5a9bbcde4c76b0f65b95f88220af32"}
Jul 07 23:20:10.571372 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:10.571344    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl" event={"ID":"312d941a-d76a-4cb8-aa34-7b4d5271bd20","Type":"ContainerStarted","Data":"3cbba2c3bcfb0f11386c3e22ef7fc3163de75b5fb51109ab085455d85c97d1ba"}
Jul 07 23:20:10.571497 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:10.571482    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl"
Jul 07 23:20:10.592100 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:10.592042    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz" podStartSLOduration=2.21134877 podStartE2EDuration="5.592027225s" totalImagesPullingTime="3.380678455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:20:10.590908018 +0000 UTC m=+395.176477380" watchObservedRunningTime="2026-07-07 23:20:10.592027225 +0000 UTC m=+395.177596585"
Jul 07 23:20:10.611108 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:10.611047    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b7668c465-5rrp6" podStartSLOduration=1.941289094 podStartE2EDuration="6.611027519s" totalImagesPullingTime="4.669738425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:20:10.610393608 +0000 UTC m=+395.195962981" watchObservedRunningTime="2026-07-07 23:20:10.611027519 +0000 UTC m=+395.196596880"
Jul 07 23:20:10.631207 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:10.631149    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl" podStartSLOduration=2.86326045 podStartE2EDuration="6.631126087s" totalImagesPullingTime="3.767865637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:20:10.630346717 +0000 UTC m=+395.215916079" watchObservedRunningTime="2026-07-07 23:20:10.631126087 +0000 UTC m=+395.216695449"
Jul 07 23:20:13.099437 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:13.099396    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:20:13.099862 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:13.099478    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:20:13.585977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:13.585937    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6" event={"ID":"757c5ad7-3cea-435f-be0e-06e9e9c91a30","Type":"ContainerStarted","Data":"edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a"}
Jul 07 23:20:13.586170 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:13.586007    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:13.611155 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:13.611103    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6" podStartSLOduration=1.980197977 podStartE2EDuration="8.611088152s" totalImagesPullingTime="6.630890175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:20:13.610691116 +0000 UTC m=+398.196260476" watchObservedRunningTime="2026-07-07 23:20:13.611088152 +0000 UTC m=+398.196657512"
Jul 07 23:20:14.524930 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.524895    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"]
Jul 07 23:20:14.528193 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.528173    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.535390 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.535368    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-46qnr\""
Jul 07 23:20:14.571943 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.571902    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"]
Jul 07 23:20:14.592591 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.592552    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:14.682358 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.682318    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.682597 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.682367    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.682597 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.682483    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.682597 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.682528    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dngw\" (UniqueName: \"kubernetes.io/projected/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-kube-api-access-7dngw\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.682800 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.682613    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.682800 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.682668    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.682800 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.682760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.682800 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.682792    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.682971 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.682886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.783633 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.783501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.783633 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.783545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.783851 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.783705    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.783851 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.783735    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.783851 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.783753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.783851 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.783785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.784053 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.783918    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dngw\" (UniqueName: \"kubernetes.io/projected/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-kube-api-access-7dngw\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.784053 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.783976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.784053 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.784007    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.784421 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.784371    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.784513 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.784473    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.784611 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.784512    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.784611 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.784545    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.784611 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.784549    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.786344 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.786312    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.786613 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.786585    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.803576 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.803540    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dngw\" (UniqueName: \"kubernetes.io/projected/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-kube-api-access-7dngw\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.805595 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.805543    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea88039b-9e95-4ccc-abeb-80a7f2fa2f72-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786np6t\" (UID: \"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:14.838901 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:14.838863    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:15.006161 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:15.006125    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"]
Jul 07 23:20:15.009456 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:20:15.009425    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea88039b_9e95_4ccc_abeb_80a7f2fa2f72.slice/crio-e46e637f4edb7b2606a9b8822dd713c959834322fb1d4f4df6e19c9b521716d2 WatchSource:0}: Error finding container e46e637f4edb7b2606a9b8822dd713c959834322fb1d4f4df6e19c9b521716d2: Status 404 returned error can't find the container with id e46e637f4edb7b2606a9b8822dd713c959834322fb1d4f4df6e19c9b521716d2
Jul 07 23:20:15.596290 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:15.596256    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t" event={"ID":"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72","Type":"ContainerStarted","Data":"e46e637f4edb7b2606a9b8822dd713c959834322fb1d4f4df6e19c9b521716d2"}
Jul 07 23:20:16.471945 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.471906    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"]
Jul 07 23:20:16.479793 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.479762    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"]
Jul 07 23:20:16.479974 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.479901    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"
Jul 07 23:20:16.483610 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.483354    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bq926\""
Jul 07 23:20:16.483610 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.483477    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"
Jul 07 23:20:16.485433 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.484540    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 07 23:20:16.487411 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.487313    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-kdv6z\""
Jul 07 23:20:16.487411 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.487399    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 07 23:20:16.489730 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.489696    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"]
Jul 07 23:20:16.498536 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.497121    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"]
Jul 07 23:20:16.605899 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.605865    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2762acc-d7af-4ca5-817e-f201d470d0ef-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k87dr\" (UID: \"b2762acc-d7af-4ca5-817e-f201d470d0ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"
Jul 07 23:20:16.606366 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.605910    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxxmb\" (UniqueName: \"kubernetes.io/projected/444865b2-ad2c-43e3-959e-a8bc5573234e-kube-api-access-gxxmb\") pod \"kserve-controller-manager-d6f4b65b8-2wdx7\" (UID: \"444865b2-ad2c-43e3-959e-a8bc5573234e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"
Jul 07 23:20:16.606366 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.605979    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v66nb\" (UniqueName: \"kubernetes.io/projected/b2762acc-d7af-4ca5-817e-f201d470d0ef-kube-api-access-v66nb\") pod \"llmisvc-controller-manager-777c6bd7d5-k87dr\" (UID: \"b2762acc-d7af-4ca5-817e-f201d470d0ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"
Jul 07 23:20:16.606366 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.606088    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/444865b2-ad2c-43e3-959e-a8bc5573234e-cert\") pod \"kserve-controller-manager-d6f4b65b8-2wdx7\" (UID: \"444865b2-ad2c-43e3-959e-a8bc5573234e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"
Jul 07 23:20:16.707148 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.707110    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2762acc-d7af-4ca5-817e-f201d470d0ef-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k87dr\" (UID: \"b2762acc-d7af-4ca5-817e-f201d470d0ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"
Jul 07 23:20:16.707148 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.707155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxxmb\" (UniqueName: \"kubernetes.io/projected/444865b2-ad2c-43e3-959e-a8bc5573234e-kube-api-access-gxxmb\") pod \"kserve-controller-manager-d6f4b65b8-2wdx7\" (UID: \"444865b2-ad2c-43e3-959e-a8bc5573234e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"
Jul 07 23:20:16.707407 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.707182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v66nb\" (UniqueName: \"kubernetes.io/projected/b2762acc-d7af-4ca5-817e-f201d470d0ef-kube-api-access-v66nb\") pod \"llmisvc-controller-manager-777c6bd7d5-k87dr\" (UID: \"b2762acc-d7af-4ca5-817e-f201d470d0ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"
Jul 07 23:20:16.707407 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.707230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/444865b2-ad2c-43e3-959e-a8bc5573234e-cert\") pod \"kserve-controller-manager-d6f4b65b8-2wdx7\" (UID: \"444865b2-ad2c-43e3-959e-a8bc5573234e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"
Jul 07 23:20:16.707407 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:20:16.707288    2578 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 07 23:20:16.707407 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:20:16.707362    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2762acc-d7af-4ca5-817e-f201d470d0ef-cert podName:b2762acc-d7af-4ca5-817e-f201d470d0ef nodeName:}" failed. No retries permitted until 2026-07-07 23:20:17.207342407 +0000 UTC m=+401.792911751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b2762acc-d7af-4ca5-817e-f201d470d0ef-cert") pod "llmisvc-controller-manager-777c6bd7d5-k87dr" (UID: "b2762acc-d7af-4ca5-817e-f201d470d0ef") : secret "llmisvc-webhook-server-cert" not found
Jul 07 23:20:16.709957 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.709932    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/444865b2-ad2c-43e3-959e-a8bc5573234e-cert\") pod \"kserve-controller-manager-d6f4b65b8-2wdx7\" (UID: \"444865b2-ad2c-43e3-959e-a8bc5573234e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"
Jul 07 23:20:16.722153 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.722086    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v66nb\" (UniqueName: \"kubernetes.io/projected/b2762acc-d7af-4ca5-817e-f201d470d0ef-kube-api-access-v66nb\") pod \"llmisvc-controller-manager-777c6bd7d5-k87dr\" (UID: \"b2762acc-d7af-4ca5-817e-f201d470d0ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"
Jul 07 23:20:16.722627 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.722608    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxxmb\" (UniqueName: \"kubernetes.io/projected/444865b2-ad2c-43e3-959e-a8bc5573234e-kube-api-access-gxxmb\") pod \"kserve-controller-manager-d6f4b65b8-2wdx7\" (UID: \"444865b2-ad2c-43e3-959e-a8bc5573234e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"
Jul 07 23:20:16.799976 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.799940    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"
Jul 07 23:20:16.980123 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:16.980098    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"]
Jul 07 23:20:16.982428 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:20:16.982394    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod444865b2_ad2c_43e3_959e_a8bc5573234e.slice/crio-8042b57613d9742096214c64a954836e8f5f87f82a2777e7ff752030ef7b5d26 WatchSource:0}: Error finding container 8042b57613d9742096214c64a954836e8f5f87f82a2777e7ff752030ef7b5d26: Status 404 returned error can't find the container with id 8042b57613d9742096214c64a954836e8f5f87f82a2777e7ff752030ef7b5d26
Jul 07 23:20:17.211960 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:17.211921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2762acc-d7af-4ca5-817e-f201d470d0ef-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k87dr\" (UID: \"b2762acc-d7af-4ca5-817e-f201d470d0ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"
Jul 07 23:20:17.214498 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:17.214473    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2762acc-d7af-4ca5-817e-f201d470d0ef-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k87dr\" (UID: \"b2762acc-d7af-4ca5-817e-f201d470d0ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"
Jul 07 23:20:17.408898 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:17.408865    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"
Jul 07 23:20:17.567545 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:17.567515    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"]
Jul 07 23:20:17.569964 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:20:17.569937    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb2762acc_d7af_4ca5_817e_f201d470d0ef.slice/crio-ffe065b125ad5e34ff80d83f50c017787217099fed8c3e854fdeb6de0355b336 WatchSource:0}: Error finding container ffe065b125ad5e34ff80d83f50c017787217099fed8c3e854fdeb6de0355b336: Status 404 returned error can't find the container with id ffe065b125ad5e34ff80d83f50c017787217099fed8c3e854fdeb6de0355b336
Jul 07 23:20:17.607103 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:17.607063    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7" event={"ID":"444865b2-ad2c-43e3-959e-a8bc5573234e","Type":"ContainerStarted","Data":"8042b57613d9742096214c64a954836e8f5f87f82a2777e7ff752030ef7b5d26"}
Jul 07 23:20:17.608315 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:17.608284    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr" event={"ID":"b2762acc-d7af-4ca5-817e-f201d470d0ef","Type":"ContainerStarted","Data":"ffe065b125ad5e34ff80d83f50c017787217099fed8c3e854fdeb6de0355b336"}
Jul 07 23:20:17.948189 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:17.948154    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"]
Jul 07 23:20:17.948366 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:17.948321    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6" podUID="757c5ad7-3cea-435f-be0e-06e9e9c91a30" containerName="discovery" containerID="cri-o://edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a" gracePeriod=30
Jul 07 23:20:18.216697 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.216668    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:18.323853 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.323809    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-kubeconfig\") pod \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") "
Jul 07 23:20:18.324037 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.323866    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-ca-configmap\") pod \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") "
Jul 07 23:20:18.324037 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.323919    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-cacerts\") pod \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") "
Jul 07 23:20:18.324037 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.323958    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tqhh\" (UniqueName: \"kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-kube-api-access-5tqhh\") pod \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") "
Jul 07 23:20:18.324037 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.323978    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-dns-cert\") pod \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") "
Jul 07 23:20:18.324467 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.324404    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "757c5ad7-3cea-435f-be0e-06e9e9c91a30" (UID: "757c5ad7-3cea-435f-be0e-06e9e9c91a30"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:20:18.324593 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.324471    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/757c5ad7-3cea-435f-be0e-06e9e9c91a30-local-certs\") pod \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") "
Jul 07 23:20:18.324593 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.324506    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-token\") pod \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\" (UID: \"757c5ad7-3cea-435f-be0e-06e9e9c91a30\") "
Jul 07 23:20:18.325244 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.324771    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-ca-configmap\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:20:18.327228 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.327182    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "757c5ad7-3cea-435f-be0e-06e9e9c91a30" (UID: "757c5ad7-3cea-435f-be0e-06e9e9c91a30"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:20:18.327490 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.327371    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-kube-api-access-5tqhh" (OuterVolumeSpecName: "kube-api-access-5tqhh") pod "757c5ad7-3cea-435f-be0e-06e9e9c91a30" (UID: "757c5ad7-3cea-435f-be0e-06e9e9c91a30"). InnerVolumeSpecName "kube-api-access-5tqhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:20:18.327490 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.327450    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "757c5ad7-3cea-435f-be0e-06e9e9c91a30" (UID: "757c5ad7-3cea-435f-be0e-06e9e9c91a30"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:20:18.327490 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.327458    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-token" (OuterVolumeSpecName: "istio-token") pod "757c5ad7-3cea-435f-be0e-06e9e9c91a30" (UID: "757c5ad7-3cea-435f-be0e-06e9e9c91a30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:20:18.327954 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.327926    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-cacerts" (OuterVolumeSpecName: "cacerts") pod "757c5ad7-3cea-435f-be0e-06e9e9c91a30" (UID: "757c5ad7-3cea-435f-be0e-06e9e9c91a30"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:20:18.328915 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.328890    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/757c5ad7-3cea-435f-be0e-06e9e9c91a30-local-certs" (OuterVolumeSpecName: "local-certs") pod "757c5ad7-3cea-435f-be0e-06e9e9c91a30" (UID: "757c5ad7-3cea-435f-be0e-06e9e9c91a30"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:20:18.426176 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.426141    2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/757c5ad7-3cea-435f-be0e-06e9e9c91a30-local-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:20:18.426176 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.426173    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:20:18.426176 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.426182    2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-kubeconfig\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:20:18.426442 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.426190    2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-cacerts\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:20:18.426442 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.426198    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tqhh\" (UniqueName: \"kubernetes.io/projected/757c5ad7-3cea-435f-be0e-06e9e9c91a30-kube-api-access-5tqhh\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:20:18.426442 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.426206    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/757c5ad7-3cea-435f-be0e-06e9e9c91a30-istio-csr-dns-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:20:18.618414 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.618321    2578 generic.go:356] "Generic (PLEG): container finished" podID="757c5ad7-3cea-435f-be0e-06e9e9c91a30" containerID="edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a" exitCode=0
Jul 07 23:20:18.618414 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.618383    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"
Jul 07 23:20:18.618414 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.618403    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6" event={"ID":"757c5ad7-3cea-435f-be0e-06e9e9c91a30","Type":"ContainerDied","Data":"edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a"}
Jul 07 23:20:18.619008 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.618451    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6" event={"ID":"757c5ad7-3cea-435f-be0e-06e9e9c91a30","Type":"ContainerDied","Data":"bb5a01f8e383e4b409bc39b42edb1eb5811491cd939c49d51bde3b6d14b96283"}
Jul 07 23:20:18.619008 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.618481    2578 scope.go:117] "RemoveContainer" containerID="edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a"
Jul 07 23:20:18.644467 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.644310    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"]
Jul 07 23:20:18.651104 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.651062    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7sfx6"]
Jul 07 23:20:18.715283 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.715257    2578 scope.go:117] "RemoveContainer" containerID="edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a"
Jul 07 23:20:18.715684 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:20:18.715663    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a\": container with ID starting with edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a not found: ID does not exist" containerID="edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a"
Jul 07 23:20:18.715748 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:18.715694    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a"} err="failed to get container status \"edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a\": rpc error: code = NotFound desc = could not find container \"edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a\": container with ID starting with edcb61e88b67121e004abba0a754f03d9099da9a6fabb85e4563910e6f65bd7a not found: ID does not exist"
Jul 07 23:20:19.962850 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:19.962815    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="757c5ad7-3cea-435f-be0e-06e9e9c91a30" path="/var/lib/kubelet/pods/757c5ad7-3cea-435f-be0e-06e9e9c91a30/volumes"
Jul 07 23:20:21.577845 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:21.577820    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-jk5xl"
Jul 07 23:20:21.579978 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:21.579956    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-vlkvz"
Jul 07 23:20:21.583944 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:21.583906    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:20:21.584015 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:21.584000    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:20:21.584056 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:21.584042    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:20:22.641050 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:22.641011    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t" event={"ID":"ea88039b-9e95-4ccc-abeb-80a7f2fa2f72","Type":"ContainerStarted","Data":"6c68734ba3a850cab9f36eb617bf0e846ce97cfd5e890b19469081d8376754f4"}
Jul 07 23:20:22.642675 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:22.642650    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7" event={"ID":"444865b2-ad2c-43e3-959e-a8bc5573234e","Type":"ContainerStarted","Data":"1b6279b30adf33f6638c459e6834ecd0cee9fb999cc73d238fc533ea1667cae0"}
Jul 07 23:20:22.642818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:22.642702    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"
Jul 07 23:20:22.644020 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:22.643996    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr" event={"ID":"b2762acc-d7af-4ca5-817e-f201d470d0ef","Type":"ContainerStarted","Data":"d97033010b24f33a01a22b9de61c38ab94f933574aa2f2086e939bc677e46c2a"}
Jul 07 23:20:22.644119 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:22.644052    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"
Jul 07 23:20:22.672018 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:22.671966    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t" podStartSLOduration=2.103199613 podStartE2EDuration="8.671947897s" totalImagesPullingTime="6.568748284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:20:22.670676433 +0000 UTC m=+407.256245798" watchObservedRunningTime="2026-07-07 23:20:22.671947897 +0000 UTC m=+407.257517257"
Jul 07 23:20:22.689826 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:22.689776    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7" podStartSLOduration=2.097961331 podStartE2EDuration="6.689758693s" totalImagesPullingTime="4.591797362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:20:22.688670296 +0000 UTC m=+407.274239659" watchObservedRunningTime="2026-07-07 23:20:22.689758693 +0000 UTC m=+407.275328054"
Jul 07 23:20:22.706492 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:22.706429    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr" podStartSLOduration=2.687995098 podStartE2EDuration="6.706408549s" totalImagesPullingTime="4.018413451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:20:22.706361721 +0000 UTC m=+407.291931082" watchObservedRunningTime="2026-07-07 23:20:22.706408549 +0000 UTC m=+407.291977910"
Jul 07 23:20:22.839420 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:22.839380    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:22.844038 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:22.844010    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:23.648517 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:23.648476    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:23.649342 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:23.649326    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786np6t"
Jul 07 23:20:53.650753 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:53.650719    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k87dr"
Jul 07 23:20:53.653934 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:20:53.653914    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2wdx7"
Jul 07 23:21:05.391217 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.391135    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp7vm"]
Jul 07 23:21:05.391671 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.391460    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="757c5ad7-3cea-435f-be0e-06e9e9c91a30" containerName="discovery"
Jul 07 23:21:05.391671 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.391470    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="757c5ad7-3cea-435f-be0e-06e9e9c91a30" containerName="discovery"
Jul 07 23:21:05.391671 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.391528    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="757c5ad7-3cea-435f-be0e-06e9e9c91a30" containerName="discovery"
Jul 07 23:21:05.394375 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.394355    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kp7vm"
Jul 07 23:21:05.397114 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.397089    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-7h6bc\""
Jul 07 23:21:05.397822 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.397796    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 23:21:05.397980 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.397959    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 23:21:05.411822 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.411778    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp7vm"]
Jul 07 23:21:05.503447 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.503403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xkjw\" (UniqueName: \"kubernetes.io/projected/06ca05da-3c40-424b-be8e-6485568eaab5-kube-api-access-9xkjw\") pod \"kuadrant-operator-catalog-kp7vm\" (UID: \"06ca05da-3c40-424b-be8e-6485568eaab5\") " pod="kuadrant-system/kuadrant-operator-catalog-kp7vm"
Jul 07 23:21:05.604759 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.604718    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xkjw\" (UniqueName: \"kubernetes.io/projected/06ca05da-3c40-424b-be8e-6485568eaab5-kube-api-access-9xkjw\") pod \"kuadrant-operator-catalog-kp7vm\" (UID: \"06ca05da-3c40-424b-be8e-6485568eaab5\") " pod="kuadrant-system/kuadrant-operator-catalog-kp7vm"
Jul 07 23:21:05.613570 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.613530    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xkjw\" (UniqueName: \"kubernetes.io/projected/06ca05da-3c40-424b-be8e-6485568eaab5-kube-api-access-9xkjw\") pod \"kuadrant-operator-catalog-kp7vm\" (UID: \"06ca05da-3c40-424b-be8e-6485568eaab5\") " pod="kuadrant-system/kuadrant-operator-catalog-kp7vm"
Jul 07 23:21:05.660943 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.660855    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp7vm"]
Jul 07 23:21:05.661149 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.661136    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kp7vm"
Jul 07 23:21:05.799897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.799832    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp7vm"]
Jul 07 23:21:05.802256 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:21:05.802233    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ca05da_3c40_424b_be8e_6485568eaab5.slice/crio-e88bac9031631d67616d1acc000bfdf5939cb3f2d13ff83a18312941465f19c9 WatchSource:0}: Error finding container e88bac9031631d67616d1acc000bfdf5939cb3f2d13ff83a18312941465f19c9: Status 404 returned error can't find the container with id e88bac9031631d67616d1acc000bfdf5939cb3f2d13ff83a18312941465f19c9
Jul 07 23:21:05.825153 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.825119    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kp7vm" event={"ID":"06ca05da-3c40-424b-be8e-6485568eaab5","Type":"ContainerStarted","Data":"e88bac9031631d67616d1acc000bfdf5939cb3f2d13ff83a18312941465f19c9"}
Jul 07 23:21:05.873812 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.873777    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h96lz"]
Jul 07 23:21:05.878124 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.878106    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h96lz"
Jul 07 23:21:05.886804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:05.886780    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h96lz"]
Jul 07 23:21:06.007773 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:06.007664    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2z7f\" (UniqueName: \"kubernetes.io/projected/bd112ae1-1905-4b0e-9af2-6a4d955ef92c-kube-api-access-b2z7f\") pod \"kuadrant-operator-catalog-h96lz\" (UID: \"bd112ae1-1905-4b0e-9af2-6a4d955ef92c\") " pod="kuadrant-system/kuadrant-operator-catalog-h96lz"
Jul 07 23:21:06.108707 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:06.108670    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2z7f\" (UniqueName: \"kubernetes.io/projected/bd112ae1-1905-4b0e-9af2-6a4d955ef92c-kube-api-access-b2z7f\") pod \"kuadrant-operator-catalog-h96lz\" (UID: \"bd112ae1-1905-4b0e-9af2-6a4d955ef92c\") " pod="kuadrant-system/kuadrant-operator-catalog-h96lz"
Jul 07 23:21:06.119772 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:06.119743    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2z7f\" (UniqueName: \"kubernetes.io/projected/bd112ae1-1905-4b0e-9af2-6a4d955ef92c-kube-api-access-b2z7f\") pod \"kuadrant-operator-catalog-h96lz\" (UID: \"bd112ae1-1905-4b0e-9af2-6a4d955ef92c\") " pod="kuadrant-system/kuadrant-operator-catalog-h96lz"
Jul 07 23:21:06.188134 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:06.188091    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h96lz"
Jul 07 23:21:06.337664 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:06.337633    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h96lz"]
Jul 07 23:21:06.340710 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:21:06.340679    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd112ae1_1905_4b0e_9af2_6a4d955ef92c.slice/crio-451ac0e72060e9883abc2b1cdd753d50c9bd9fad27ddb2fe4445bf915f8034c2 WatchSource:0}: Error finding container 451ac0e72060e9883abc2b1cdd753d50c9bd9fad27ddb2fe4445bf915f8034c2: Status 404 returned error can't find the container with id 451ac0e72060e9883abc2b1cdd753d50c9bd9fad27ddb2fe4445bf915f8034c2
Jul 07 23:21:06.833998 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:06.833957    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h96lz" event={"ID":"bd112ae1-1905-4b0e-9af2-6a4d955ef92c","Type":"ContainerStarted","Data":"451ac0e72060e9883abc2b1cdd753d50c9bd9fad27ddb2fe4445bf915f8034c2"}
Jul 07 23:21:08.846875 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:08.846835    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kp7vm" event={"ID":"06ca05da-3c40-424b-be8e-6485568eaab5","Type":"ContainerStarted","Data":"e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e"}
Jul 07 23:21:08.847382 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:08.846890    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-kp7vm" podUID="06ca05da-3c40-424b-be8e-6485568eaab5" containerName="registry-server" containerID="cri-o://e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e" gracePeriod=2
Jul 07 23:21:08.848375 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:08.848351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h96lz" event={"ID":"bd112ae1-1905-4b0e-9af2-6a4d955ef92c","Type":"ContainerStarted","Data":"2895e7c4dc5d6b6bca0ea949e6587a1e1fd367e054c7331927cca7328701c630"}
Jul 07 23:21:08.863640 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:08.863540    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kp7vm" podStartSLOduration=1.7029205360000002 podStartE2EDuration="3.863522927s" totalImagesPullingTime="2.160602391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:21:08.862744416 +0000 UTC m=+453.448313778" watchObservedRunningTime="2026-07-07 23:21:08.863522927 +0000 UTC m=+453.449092287"
Jul 07 23:21:08.881610 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:08.881538    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h96lz" podStartSLOduration=2.258284227 podStartE2EDuration="3.881520944s" totalImagesPullingTime="1.623236717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:21:08.880034235 +0000 UTC m=+453.465603596" watchObservedRunningTime="2026-07-07 23:21:08.881520944 +0000 UTC m=+453.467090305"
Jul 07 23:21:09.097434 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.097367    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kp7vm"
Jul 07 23:21:09.238908 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.238871    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xkjw\" (UniqueName: \"kubernetes.io/projected/06ca05da-3c40-424b-be8e-6485568eaab5-kube-api-access-9xkjw\") pod \"06ca05da-3c40-424b-be8e-6485568eaab5\" (UID: \"06ca05da-3c40-424b-be8e-6485568eaab5\") "
Jul 07 23:21:09.241065 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.241030    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06ca05da-3c40-424b-be8e-6485568eaab5-kube-api-access-9xkjw" (OuterVolumeSpecName: "kube-api-access-9xkjw") pod "06ca05da-3c40-424b-be8e-6485568eaab5" (UID: "06ca05da-3c40-424b-be8e-6485568eaab5"). InnerVolumeSpecName "kube-api-access-9xkjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:21:09.339677 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.339643    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xkjw\" (UniqueName: \"kubernetes.io/projected/06ca05da-3c40-424b-be8e-6485568eaab5-kube-api-access-9xkjw\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:21:09.858696 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.858659    2578 generic.go:356] "Generic (PLEG): container finished" podID="06ca05da-3c40-424b-be8e-6485568eaab5" containerID="e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e" exitCode=0
Jul 07 23:21:09.859137 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.858719    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kp7vm"
Jul 07 23:21:09.859137 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.858748    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kp7vm" event={"ID":"06ca05da-3c40-424b-be8e-6485568eaab5","Type":"ContainerDied","Data":"e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e"}
Jul 07 23:21:09.859137 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.858823    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kp7vm" event={"ID":"06ca05da-3c40-424b-be8e-6485568eaab5","Type":"ContainerDied","Data":"e88bac9031631d67616d1acc000bfdf5939cb3f2d13ff83a18312941465f19c9"}
Jul 07 23:21:09.859137 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.858847    2578 scope.go:117] "RemoveContainer" containerID="e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e"
Jul 07 23:21:09.868807 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.868786    2578 scope.go:117] "RemoveContainer" containerID="e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e"
Jul 07 23:21:09.869127 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:21:09.869108    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e\": container with ID starting with e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e not found: ID does not exist" containerID="e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e"
Jul 07 23:21:09.869207 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.869139    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e"} err="failed to get container status \"e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e\": rpc error: code = NotFound desc = could not find container \"e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e\": container with ID starting with e4ef37bb53ee68a2fc29697cce5167ac3f08702b9b19e82d45c94e48d54ba75e not found: ID does not exist"
Jul 07 23:21:09.879770 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.879740    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp7vm"]
Jul 07 23:21:09.883725 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.883697    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp7vm"]
Jul 07 23:21:09.962736 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:09.962698    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06ca05da-3c40-424b-be8e-6485568eaab5" path="/var/lib/kubelet/pods/06ca05da-3c40-424b-be8e-6485568eaab5/volumes"
Jul 07 23:21:16.189141 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:16.189101    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-h96lz"
Jul 07 23:21:16.189743 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:16.189203    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-h96lz"
Jul 07 23:21:16.211947 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:16.211922    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-h96lz"
Jul 07 23:21:16.913267 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:16.913236    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-h96lz"
Jul 07 23:21:36.664908 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.664865    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69745cc477-mf458"]
Jul 07 23:21:36.665369 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.665157    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ca05da-3c40-424b-be8e-6485568eaab5" containerName="registry-server"
Jul 07 23:21:36.665369 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.665168    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ca05da-3c40-424b-be8e-6485568eaab5" containerName="registry-server"
Jul 07 23:21:36.665369 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.665232    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="06ca05da-3c40-424b-be8e-6485568eaab5" containerName="registry-server"
Jul 07 23:21:36.669263 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.669244    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.674375 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.674340    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m6lx8\""
Jul 07 23:21:36.674540 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.674404    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 23:21:36.674540 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.674438    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 23:21:36.675072 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.675051    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 23:21:36.675200 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.675087    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 23:21:36.675310 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.675295    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 23:21:36.675365 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.675335    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 23:21:36.679690 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.679672    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 23:21:36.687135 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.687107    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69745cc477-mf458"]
Jul 07 23:21:36.690161 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.690134    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 23:21:36.774791 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.774754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-service-ca\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.774791 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.774791    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/312bb119-7af4-4263-9d75-f5131a4ae8ee-console-serving-cert\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.775039 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.774829    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-console-config\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.775039 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.774882    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/312bb119-7af4-4263-9d75-f5131a4ae8ee-console-oauth-config\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.775039 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.774943    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvf7h\" (UniqueName: \"kubernetes.io/projected/312bb119-7af4-4263-9d75-f5131a4ae8ee-kube-api-access-nvf7h\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.775039 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.774978    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-trusted-ca-bundle\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.775039 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.775000    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-oauth-serving-cert\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.876085 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.876044    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-service-ca\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.876085 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.876083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/312bb119-7af4-4263-9d75-f5131a4ae8ee-console-serving-cert\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.876314 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.876214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-console-config\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.876314 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.876268    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/312bb119-7af4-4263-9d75-f5131a4ae8ee-console-oauth-config\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.876391 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.876314    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvf7h\" (UniqueName: \"kubernetes.io/projected/312bb119-7af4-4263-9d75-f5131a4ae8ee-kube-api-access-nvf7h\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.876391 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.876352    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-trusted-ca-bundle\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.876391 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.876387    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-oauth-serving-cert\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.876853 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.876816    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-service-ca\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.876964 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.876941    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-console-config\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.877058 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.877044    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-oauth-serving-cert\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.877311 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.877290    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312bb119-7af4-4263-9d75-f5131a4ae8ee-trusted-ca-bundle\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.878709 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.878683    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/312bb119-7af4-4263-9d75-f5131a4ae8ee-console-oauth-config\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.878797 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.878787    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/312bb119-7af4-4263-9d75-f5131a4ae8ee-console-serving-cert\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.886537 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.886514    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvf7h\" (UniqueName: \"kubernetes.io/projected/312bb119-7af4-4263-9d75-f5131a4ae8ee-kube-api-access-nvf7h\") pod \"console-69745cc477-mf458\" (UID: \"312bb119-7af4-4263-9d75-f5131a4ae8ee\") " pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:36.978977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:36.978886    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:37.121162 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:37.121137    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69745cc477-mf458"]
Jul 07 23:21:37.123236 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:21:37.123204    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312bb119_7af4_4263_9d75_f5131a4ae8ee.slice/crio-cde041623c15b18eca3e7d023707992847e2028d0d323b8ab94502a567ff40b0 WatchSource:0}: Error finding container cde041623c15b18eca3e7d023707992847e2028d0d323b8ab94502a567ff40b0: Status 404 returned error can't find the container with id cde041623c15b18eca3e7d023707992847e2028d0d323b8ab94502a567ff40b0
Jul 07 23:21:37.985350 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:37.985309    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69745cc477-mf458" event={"ID":"312bb119-7af4-4263-9d75-f5131a4ae8ee","Type":"ContainerStarted","Data":"92d9fb84fcf59c479ee6a3bdd152f9ecb73a72e380be172503965a210ee3ff6f"}
Jul 07 23:21:37.985819 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:37.985365    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69745cc477-mf458" event={"ID":"312bb119-7af4-4263-9d75-f5131a4ae8ee","Type":"ContainerStarted","Data":"cde041623c15b18eca3e7d023707992847e2028d0d323b8ab94502a567ff40b0"}
Jul 07 23:21:38.006805 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.006746    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69745cc477-mf458" podStartSLOduration=2.006729741 podStartE2EDuration="2.006729741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:21:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:21:38.005744403 +0000 UTC m=+482.591313764" watchObservedRunningTime="2026-07-07 23:21:38.006729741 +0000 UTC m=+482.592299102"
Jul 07 23:21:38.506672 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.506625    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt"]
Jul 07 23:21:38.509935 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.509916    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt"
Jul 07 23:21:38.512474 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.512446    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 07 23:21:38.512697 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.512678    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jg9ft\""
Jul 07 23:21:38.521256 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.521223    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt"]
Jul 07 23:21:38.693869 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.693822    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk8zq\" (UniqueName: \"kubernetes.io/projected/de987442-d98a-403c-9a1b-ed2a61c2fd02-kube-api-access-nk8zq\") pod \"dns-operator-controller-manager-864dd78c4c-hpnqt\" (UID: \"de987442-d98a-403c-9a1b-ed2a61c2fd02\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt"
Jul 07 23:21:38.795330 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.795241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk8zq\" (UniqueName: \"kubernetes.io/projected/de987442-d98a-403c-9a1b-ed2a61c2fd02-kube-api-access-nk8zq\") pod \"dns-operator-controller-manager-864dd78c4c-hpnqt\" (UID: \"de987442-d98a-403c-9a1b-ed2a61c2fd02\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt"
Jul 07 23:21:38.806598 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.806539    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk8zq\" (UniqueName: \"kubernetes.io/projected/de987442-d98a-403c-9a1b-ed2a61c2fd02-kube-api-access-nk8zq\") pod \"dns-operator-controller-manager-864dd78c4c-hpnqt\" (UID: \"de987442-d98a-403c-9a1b-ed2a61c2fd02\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt"
Jul 07 23:21:38.821320 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.821287    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt"
Jul 07 23:21:38.968571 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.968484    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt"]
Jul 07 23:21:38.971329 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:21:38.971299    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde987442_d98a_403c_9a1b_ed2a61c2fd02.slice/crio-77560e6e61fb2bd328cd3f4666480b5ff0c6e26e93fbdd9521f31e0eba9174eb WatchSource:0}: Error finding container 77560e6e61fb2bd328cd3f4666480b5ff0c6e26e93fbdd9521f31e0eba9174eb: Status 404 returned error can't find the container with id 77560e6e61fb2bd328cd3f4666480b5ff0c6e26e93fbdd9521f31e0eba9174eb
Jul 07 23:21:38.991609 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:38.991550    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt" event={"ID":"de987442-d98a-403c-9a1b-ed2a61c2fd02","Type":"ContainerStarted","Data":"77560e6e61fb2bd328cd3f4666480b5ff0c6e26e93fbdd9521f31e0eba9174eb"}
Jul 07 23:21:41.003431 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:41.003395    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt" event={"ID":"de987442-d98a-403c-9a1b-ed2a61c2fd02","Type":"ContainerStarted","Data":"4b7eb95fdf284151cd26b514888c06bbbca8ed982e714d68be4152b666048414"}
Jul 07 23:21:41.003933 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:41.003493    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt"
Jul 07 23:21:41.075472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:41.075368    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt" podStartSLOduration=1.269246751 podStartE2EDuration="3.075352355s" totalImagesPullingTime="1.806105604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:21:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:21:41.069292068 +0000 UTC m=+485.654861407" watchObservedRunningTime="2026-07-07 23:21:41.075352355 +0000 UTC m=+485.660921716"
Jul 07 23:21:44.059392 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.059355    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"]
Jul 07 23:21:44.066409 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.066386    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:21:44.069827 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.069804    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-m2xpk\""
Jul 07 23:21:44.090678 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.090638    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"]
Jul 07 23:21:44.145548 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.145510    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnjsj\" (UniqueName: \"kubernetes.io/projected/67e28c09-4f93-472d-affb-dfa7c2a23048-kube-api-access-pnjsj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" (UID: \"67e28c09-4f93-472d-affb-dfa7c2a23048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:21:44.145763 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.145609    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/67e28c09-4f93-472d-affb-dfa7c2a23048-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" (UID: \"67e28c09-4f93-472d-affb-dfa7c2a23048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:21:44.246610 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.246536    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnjsj\" (UniqueName: \"kubernetes.io/projected/67e28c09-4f93-472d-affb-dfa7c2a23048-kube-api-access-pnjsj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" (UID: \"67e28c09-4f93-472d-affb-dfa7c2a23048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:21:44.246834 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.246640    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/67e28c09-4f93-472d-affb-dfa7c2a23048-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" (UID: \"67e28c09-4f93-472d-affb-dfa7c2a23048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:21:44.246972 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.246953    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/67e28c09-4f93-472d-affb-dfa7c2a23048-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" (UID: \"67e28c09-4f93-472d-affb-dfa7c2a23048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:21:44.263909 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.263881    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnjsj\" (UniqueName: \"kubernetes.io/projected/67e28c09-4f93-472d-affb-dfa7c2a23048-kube-api-access-pnjsj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" (UID: \"67e28c09-4f93-472d-affb-dfa7c2a23048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:21:44.377195 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.377161    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:21:44.527392 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:44.527368    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"]
Jul 07 23:21:44.530411 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:21:44.530384    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e28c09_4f93_472d_affb_dfa7c2a23048.slice/crio-f1abad4dbf55d95bc7979b817c667e7480b77638a09bbf66e3f281da0ef068dc WatchSource:0}: Error finding container f1abad4dbf55d95bc7979b817c667e7480b77638a09bbf66e3f281da0ef068dc: Status 404 returned error can't find the container with id f1abad4dbf55d95bc7979b817c667e7480b77638a09bbf66e3f281da0ef068dc
Jul 07 23:21:45.021794 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:45.021759    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9" event={"ID":"67e28c09-4f93-472d-affb-dfa7c2a23048","Type":"ContainerStarted","Data":"f1abad4dbf55d95bc7979b817c667e7480b77638a09bbf66e3f281da0ef068dc"}
Jul 07 23:21:46.979863 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:46.979822    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:46.980259 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:46.979917    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:46.984653 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:46.984632    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:47.035425 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:47.035398    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69745cc477-mf458"
Jul 07 23:21:47.129502 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:47.129471    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"]
Jul 07 23:21:47.133136 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:47.133109    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:21:47.137054 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:47.137029    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-xcj8q\""
Jul 07 23:21:47.153307 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:47.153272    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"]
Jul 07 23:21:47.171921 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:47.171881    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzb6b\" (UniqueName: \"kubernetes.io/projected/69b1fc41-cff2-47d8-aafc-c852e65ea4f8-kube-api-access-bzb6b\") pod \"limitador-operator-controller-manager-859d7b78bd-tw8mt\" (UID: \"69b1fc41-cff2-47d8-aafc-c852e65ea4f8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:21:47.273239 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:47.273124    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzb6b\" (UniqueName: \"kubernetes.io/projected/69b1fc41-cff2-47d8-aafc-c852e65ea4f8-kube-api-access-bzb6b\") pod \"limitador-operator-controller-manager-859d7b78bd-tw8mt\" (UID: \"69b1fc41-cff2-47d8-aafc-c852e65ea4f8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:21:47.314554 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:47.314521    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzb6b\" (UniqueName: \"kubernetes.io/projected/69b1fc41-cff2-47d8-aafc-c852e65ea4f8-kube-api-access-bzb6b\") pod \"limitador-operator-controller-manager-859d7b78bd-tw8mt\" (UID: \"69b1fc41-cff2-47d8-aafc-c852e65ea4f8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:21:47.444019 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:47.443979    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:21:47.657026 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:47.657003    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"]
Jul 07 23:21:47.659234 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:21:47.659206    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b1fc41_cff2_47d8_aafc_c852e65ea4f8.slice/crio-eaa3cb3391b9707e65f647b7e4bc06f38a790d8aaaa1e155131ed1f69a61ffa4 WatchSource:0}: Error finding container eaa3cb3391b9707e65f647b7e4bc06f38a790d8aaaa1e155131ed1f69a61ffa4: Status 404 returned error can't find the container with id eaa3cb3391b9707e65f647b7e4bc06f38a790d8aaaa1e155131ed1f69a61ffa4
Jul 07 23:21:48.035438 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:48.035405    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt" event={"ID":"69b1fc41-cff2-47d8-aafc-c852e65ea4f8","Type":"ContainerStarted","Data":"eaa3cb3391b9707e65f647b7e4bc06f38a790d8aaaa1e155131ed1f69a61ffa4"}
Jul 07 23:21:50.047660 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:50.047624    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt" event={"ID":"69b1fc41-cff2-47d8-aafc-c852e65ea4f8","Type":"ContainerStarted","Data":"c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621"}
Jul 07 23:21:50.048115 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:50.047746    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:21:50.069954 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:50.069899    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt" podStartSLOduration=1.262491425 podStartE2EDuration="3.06988113s" totalImagesPullingTime="1.807389705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:21:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:21:50.068706383 +0000 UTC m=+494.654275745" watchObservedRunningTime="2026-07-07 23:21:50.06988113 +0000 UTC m=+494.655450488"
Jul 07 23:21:50.572963 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:50.572928    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hhf48"]
Jul 07 23:21:50.576590 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:50.576550    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hhf48"
Jul 07 23:21:50.579677 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:50.579655    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kgbsj\""
Jul 07 23:21:50.588177 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:50.588148    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hhf48"]
Jul 07 23:21:50.703050 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:50.702999    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh77z\" (UniqueName: \"kubernetes.io/projected/f7aae87a-c894-43c1-9426-d164ec5c391a-kube-api-access-rh77z\") pod \"authorino-operator-75cc8d5cb7-hhf48\" (UID: \"f7aae87a-c894-43c1-9426-d164ec5c391a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hhf48"
Jul 07 23:21:50.803832 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:50.803789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh77z\" (UniqueName: \"kubernetes.io/projected/f7aae87a-c894-43c1-9426-d164ec5c391a-kube-api-access-rh77z\") pod \"authorino-operator-75cc8d5cb7-hhf48\" (UID: \"f7aae87a-c894-43c1-9426-d164ec5c391a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hhf48"
Jul 07 23:21:50.813938 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:50.813897    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh77z\" (UniqueName: \"kubernetes.io/projected/f7aae87a-c894-43c1-9426-d164ec5c391a-kube-api-access-rh77z\") pod \"authorino-operator-75cc8d5cb7-hhf48\" (UID: \"f7aae87a-c894-43c1-9426-d164ec5c391a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hhf48"
Jul 07 23:21:50.889309 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:50.889266    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hhf48"
Jul 07 23:21:51.193751 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:51.193726    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hhf48"]
Jul 07 23:21:51.196649 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:21:51.196620    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7aae87a_c894_43c1_9426_d164ec5c391a.slice/crio-38e47e642f52f69bc33675bbee61c3b722cff41a0aa5470cfd3b0dd34971856f WatchSource:0}: Error finding container 38e47e642f52f69bc33675bbee61c3b722cff41a0aa5470cfd3b0dd34971856f: Status 404 returned error can't find the container with id 38e47e642f52f69bc33675bbee61c3b722cff41a0aa5470cfd3b0dd34971856f
Jul 07 23:21:52.009747 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:52.009712    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpnqt"
Jul 07 23:21:52.062828 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:52.062790    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9" event={"ID":"67e28c09-4f93-472d-affb-dfa7c2a23048","Type":"ContainerStarted","Data":"a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd"}
Jul 07 23:21:52.063843 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:52.063322    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:21:52.065144 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:52.065075    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hhf48" event={"ID":"f7aae87a-c894-43c1-9426-d164ec5c391a","Type":"ContainerStarted","Data":"38e47e642f52f69bc33675bbee61c3b722cff41a0aa5470cfd3b0dd34971856f"}
Jul 07 23:21:52.089084 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:52.089024    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9" podStartSLOduration=1.511366781 podStartE2EDuration="8.089004648s" totalImagesPullingTime="6.577637867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:21:52.087426379 +0000 UTC m=+496.672995740" watchObservedRunningTime="2026-07-07 23:21:52.089004648 +0000 UTC m=+496.674574014"
Jul 07 23:21:54.078894 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:54.078852    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hhf48" event={"ID":"f7aae87a-c894-43c1-9426-d164ec5c391a","Type":"ContainerStarted","Data":"0855afcb0b215cc63a3b1702d8789af8a06b874c70bdfd766a94028f24b996e6"}
Jul 07 23:21:54.079355 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:54.078939    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hhf48"
Jul 07 23:21:54.102172 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:21:54.102120    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hhf48" podStartSLOduration=1.834357944 podStartE2EDuration="4.102104586s" totalImagesPullingTime="2.267746642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:21:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:21:54.101081228 +0000 UTC m=+498.686650589" watchObservedRunningTime="2026-07-07 23:21:54.102104586 +0000 UTC m=+498.687673947"
Jul 07 23:22:01.056478 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:01.056446    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:22:03.074327 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:03.074298    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:22:05.079888 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.079798    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"]
Jul 07 23:22:05.080286 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.080000    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" containerName="manager" containerID="cri-o://a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd" gracePeriod=2
Jul 07 23:22:05.085833 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.085803    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hhf48"
Jul 07 23:22:05.086373 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.086299    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"]
Jul 07 23:22:05.108697 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.108666    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"]
Jul 07 23:22:05.108897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.108865    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt" podUID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" containerName="manager" containerID="cri-o://c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621" gracePeriod=2
Jul 07 23:22:05.121238 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.121203    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"]
Jul 07 23:22:05.130256 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.130226    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"]
Jul 07 23:22:05.130766 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.130749    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" containerName="manager"
Jul 07 23:22:05.130806 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.130770    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" containerName="manager"
Jul 07 23:22:05.130806 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.130782    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" containerName="manager"
Jul 07 23:22:05.130806 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.130791    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" containerName="manager"
Jul 07 23:22:05.130912 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.130869    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" containerName="manager"
Jul 07 23:22:05.130912 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.130883    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" containerName="manager"
Jul 07 23:22:05.131128 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.131100    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:22:05.134131 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.134110    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:05.135985 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.135960    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:22:05.139192 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.139141    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tw8mt\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:22:05.141744 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.141713    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:22:05.157267 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.157234    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc"]
Jul 07 23:22:05.160606 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.160578    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"]
Jul 07 23:22:05.160734 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.160655    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc"
Jul 07 23:22:05.172073 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.172040    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc"]
Jul 07 23:22:05.177847 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.177806    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tw8mt\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:22:05.180624 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.180589    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:22:05.202834 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.202796    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tw8mt\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:22:05.235219 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.235180    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz2tv\" (UniqueName: \"kubernetes.io/projected/868a944f-a78a-4d16-835b-a9fad055beb9-kube-api-access-qz2tv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dpkpc\" (UID: \"868a944f-a78a-4d16-835b-a9fad055beb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:05.235404 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.235270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/868a944f-a78a-4d16-835b-a9fad055beb9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dpkpc\" (UID: \"868a944f-a78a-4d16-835b-a9fad055beb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:05.235404 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.235293    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lthr9\" (UniqueName: \"kubernetes.io/projected/0e26a7d9-fd80-49b0-af32-35d646136692-kube-api-access-lthr9\") pod \"limitador-operator-controller-manager-859d7b78bd-4dkdc\" (UID: \"0e26a7d9-fd80-49b0-af32-35d646136692\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc"
Jul 07 23:22:05.336587 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.336328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/868a944f-a78a-4d16-835b-a9fad055beb9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dpkpc\" (UID: \"868a944f-a78a-4d16-835b-a9fad055beb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:05.336587 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.336370    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lthr9\" (UniqueName: \"kubernetes.io/projected/0e26a7d9-fd80-49b0-af32-35d646136692-kube-api-access-lthr9\") pod \"limitador-operator-controller-manager-859d7b78bd-4dkdc\" (UID: \"0e26a7d9-fd80-49b0-af32-35d646136692\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc"
Jul 07 23:22:05.336587 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.336399    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz2tv\" (UniqueName: \"kubernetes.io/projected/868a944f-a78a-4d16-835b-a9fad055beb9-kube-api-access-qz2tv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dpkpc\" (UID: \"868a944f-a78a-4d16-835b-a9fad055beb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:05.336845 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.336822    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/868a944f-a78a-4d16-835b-a9fad055beb9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dpkpc\" (UID: \"868a944f-a78a-4d16-835b-a9fad055beb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:05.353933 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.353901    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:22:05.356493 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.356458    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:22:05.357041 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.357020    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:22:05.357337 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.357315    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz2tv\" (UniqueName: \"kubernetes.io/projected/868a944f-a78a-4d16-835b-a9fad055beb9-kube-api-access-qz2tv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dpkpc\" (UID: \"868a944f-a78a-4d16-835b-a9fad055beb9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:05.358054 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.358037    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lthr9\" (UniqueName: \"kubernetes.io/projected/0e26a7d9-fd80-49b0-af32-35d646136692-kube-api-access-lthr9\") pod \"limitador-operator-controller-manager-859d7b78bd-4dkdc\" (UID: \"0e26a7d9-fd80-49b0-af32-35d646136692\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc"
Jul 07 23:22:05.358713 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.358688    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tw8mt\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:22:05.361148 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.361116    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:22:05.363275 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.363251    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tw8mt\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:22:05.437170 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.437126    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzb6b\" (UniqueName: \"kubernetes.io/projected/69b1fc41-cff2-47d8-aafc-c852e65ea4f8-kube-api-access-bzb6b\") pod \"69b1fc41-cff2-47d8-aafc-c852e65ea4f8\" (UID: \"69b1fc41-cff2-47d8-aafc-c852e65ea4f8\") "
Jul 07 23:22:05.437367 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.437210    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/67e28c09-4f93-472d-affb-dfa7c2a23048-extensions-socket-volume\") pod \"67e28c09-4f93-472d-affb-dfa7c2a23048\" (UID: \"67e28c09-4f93-472d-affb-dfa7c2a23048\") "
Jul 07 23:22:05.437367 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.437243    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnjsj\" (UniqueName: \"kubernetes.io/projected/67e28c09-4f93-472d-affb-dfa7c2a23048-kube-api-access-pnjsj\") pod \"67e28c09-4f93-472d-affb-dfa7c2a23048\" (UID: \"67e28c09-4f93-472d-affb-dfa7c2a23048\") "
Jul 07 23:22:05.437744 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.437709    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e28c09-4f93-472d-affb-dfa7c2a23048-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "67e28c09-4f93-472d-affb-dfa7c2a23048" (UID: "67e28c09-4f93-472d-affb-dfa7c2a23048"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:22:05.439261 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.439241    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e28c09-4f93-472d-affb-dfa7c2a23048-kube-api-access-pnjsj" (OuterVolumeSpecName: "kube-api-access-pnjsj") pod "67e28c09-4f93-472d-affb-dfa7c2a23048" (UID: "67e28c09-4f93-472d-affb-dfa7c2a23048"). InnerVolumeSpecName "kube-api-access-pnjsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:22:05.439328 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.439267    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69b1fc41-cff2-47d8-aafc-c852e65ea4f8-kube-api-access-bzb6b" (OuterVolumeSpecName: "kube-api-access-bzb6b") pod "69b1fc41-cff2-47d8-aafc-c852e65ea4f8" (UID: "69b1fc41-cff2-47d8-aafc-c852e65ea4f8"). InnerVolumeSpecName "kube-api-access-bzb6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:22:05.507694 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.507654    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:05.513937 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.513909    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc"
Jul 07 23:22:05.538292 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.538237    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/67e28c09-4f93-472d-affb-dfa7c2a23048-extensions-socket-volume\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:22:05.538292 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.538262    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnjsj\" (UniqueName: \"kubernetes.io/projected/67e28c09-4f93-472d-affb-dfa7c2a23048-kube-api-access-pnjsj\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:22:05.538292 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.538274    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzb6b\" (UniqueName: \"kubernetes.io/projected/69b1fc41-cff2-47d8-aafc-c852e65ea4f8-kube-api-access-bzb6b\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:22:05.671299 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.671267    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"]
Jul 07 23:22:05.674883 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:22:05.674848    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod868a944f_a78a_4d16_835b_a9fad055beb9.slice/crio-39bfa64dffcd397ea641272ac03ac0147fe17cf403107ffb508a92e6ab24f77d WatchSource:0}: Error finding container 39bfa64dffcd397ea641272ac03ac0147fe17cf403107ffb508a92e6ab24f77d: Status 404 returned error can't find the container with id 39bfa64dffcd397ea641272ac03ac0147fe17cf403107ffb508a92e6ab24f77d
Jul 07 23:22:05.691961 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.691930    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc"]
Jul 07 23:22:05.696022 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:22:05.695996    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e26a7d9_fd80_49b0_af32_35d646136692.slice/crio-d0956b8170b0f7c4ac1fc92331ce3fd26a5b5ec64320ddc277cd73f44ee58f40 WatchSource:0}: Error finding container d0956b8170b0f7c4ac1fc92331ce3fd26a5b5ec64320ddc277cd73f44ee58f40: Status 404 returned error can't find the container with id d0956b8170b0f7c4ac1fc92331ce3fd26a5b5ec64320ddc277cd73f44ee58f40
Jul 07 23:22:05.964716 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.964677    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nzvr9\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:22:05.967114 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.967087    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67e28c09-4f93-472d-affb-dfa7c2a23048" path="/var/lib/kubelet/pods/67e28c09-4f93-472d-affb-dfa7c2a23048/volumes"
Jul 07 23:22:05.967203 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:05.967182    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tw8mt\" is forbidden: User \"system:node:ip-10-0-137-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-77.ec2.internal' and this object" podUID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:22:05.967407 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:05.967394    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" path="/var/lib/kubelet/pods/69b1fc41-cff2-47d8-aafc-c852e65ea4f8/volumes"
Jul 07 23:22:06.134383 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.134337    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc" event={"ID":"0e26a7d9-fd80-49b0-af32-35d646136692","Type":"ContainerStarted","Data":"7b6a2add4cb38bc741f03af8f4190ebeaad074064ab67d1cf2c2613dc05db73b"}
Jul 07 23:22:06.134833 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.134405    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc" event={"ID":"0e26a7d9-fd80-49b0-af32-35d646136692","Type":"ContainerStarted","Data":"d0956b8170b0f7c4ac1fc92331ce3fd26a5b5ec64320ddc277cd73f44ee58f40"}
Jul 07 23:22:06.134833 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.134432    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc"
Jul 07 23:22:06.135798 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.135764    2578 generic.go:356] "Generic (PLEG): container finished" podID="69b1fc41-cff2-47d8-aafc-c852e65ea4f8" containerID="c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621" exitCode=0
Jul 07 23:22:06.135883 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.135814    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tw8mt"
Jul 07 23:22:06.135883 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.135870    2578 scope.go:117] "RemoveContainer" containerID="c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621"
Jul 07 23:22:06.137771 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.137671    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc" event={"ID":"868a944f-a78a-4d16-835b-a9fad055beb9","Type":"ContainerStarted","Data":"ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426"}
Jul 07 23:22:06.137771 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.137726    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc" event={"ID":"868a944f-a78a-4d16-835b-a9fad055beb9","Type":"ContainerStarted","Data":"39bfa64dffcd397ea641272ac03ac0147fe17cf403107ffb508a92e6ab24f77d"}
Jul 07 23:22:06.137935 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.137813    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:06.139583 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.139532    2578 generic.go:356] "Generic (PLEG): container finished" podID="67e28c09-4f93-472d-affb-dfa7c2a23048" containerID="a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd" exitCode=0
Jul 07 23:22:06.139693 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.139611    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nzvr9"
Jul 07 23:22:06.145932 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.145898    2578 scope.go:117] "RemoveContainer" containerID="c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621"
Jul 07 23:22:06.146230 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:06.146206    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621\": container with ID starting with c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621 not found: ID does not exist" containerID="c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621"
Jul 07 23:22:06.146304 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.146244    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621"} err="failed to get container status \"c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621\": rpc error: code = NotFound desc = could not find container \"c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621\": container with ID starting with c133c0ac66bca55c1963ffe3c3d119cb963b5079a5ee4e7e13de20ed3cf07621 not found: ID does not exist"
Jul 07 23:22:06.146304 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.146269    2578 scope.go:117] "RemoveContainer" containerID="a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd"
Jul 07 23:22:06.154522 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.154501    2578 scope.go:117] "RemoveContainer" containerID="a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd"
Jul 07 23:22:06.154846 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:06.154826    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd\": container with ID starting with a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd not found: ID does not exist" containerID="a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd"
Jul 07 23:22:06.154902 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.154854    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd"} err="failed to get container status \"a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd\": rpc error: code = NotFound desc = could not find container \"a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd\": container with ID starting with a3a56bcfc155e6e3180a7ce2e8beccd1d3bce599ad6b153e419cb7ed81b19afd not found: ID does not exist"
Jul 07 23:22:06.158256 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.158209    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc" podStartSLOduration=1.1581938489999999 podStartE2EDuration="1.158193849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:22:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:22:06.157739129 +0000 UTC m=+510.743308491" watchObservedRunningTime="2026-07-07 23:22:06.158193849 +0000 UTC m=+510.743763209"
Jul 07 23:22:06.213096 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:06.213048    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc" podStartSLOduration=1.213029044 podStartE2EDuration="1.213029044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:22:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:22:06.210904034 +0000 UTC m=+510.796473395" watchObservedRunningTime="2026-07-07 23:22:06.213029044 +0000 UTC m=+510.798598405"
Jul 07 23:22:17.147918 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:17.147883    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:17.148495 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:17.147953    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dkdc"
Jul 07 23:22:23.299337 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:23.299298    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"]
Jul 07 23:22:23.299772 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:23.299504    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc" podUID="868a944f-a78a-4d16-835b-a9fad055beb9" containerName="manager" containerID="cri-o://ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426" gracePeriod=10
Jul 07 23:22:23.554476 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:23.554402    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:23.587156 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:23.587126    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz2tv\" (UniqueName: \"kubernetes.io/projected/868a944f-a78a-4d16-835b-a9fad055beb9-kube-api-access-qz2tv\") pod \"868a944f-a78a-4d16-835b-a9fad055beb9\" (UID: \"868a944f-a78a-4d16-835b-a9fad055beb9\") "
Jul 07 23:22:23.589127 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:23.589092    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/868a944f-a78a-4d16-835b-a9fad055beb9-kube-api-access-qz2tv" (OuterVolumeSpecName: "kube-api-access-qz2tv") pod "868a944f-a78a-4d16-835b-a9fad055beb9" (UID: "868a944f-a78a-4d16-835b-a9fad055beb9"). InnerVolumeSpecName "kube-api-access-qz2tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:22:23.688429 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:23.688390    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/868a944f-a78a-4d16-835b-a9fad055beb9-extensions-socket-volume\") pod \"868a944f-a78a-4d16-835b-a9fad055beb9\" (UID: \"868a944f-a78a-4d16-835b-a9fad055beb9\") "
Jul 07 23:22:23.688637 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:23.688550    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz2tv\" (UniqueName: \"kubernetes.io/projected/868a944f-a78a-4d16-835b-a9fad055beb9-kube-api-access-qz2tv\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:22:23.688770 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:23.688746    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/868a944f-a78a-4d16-835b-a9fad055beb9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "868a944f-a78a-4d16-835b-a9fad055beb9" (UID: "868a944f-a78a-4d16-835b-a9fad055beb9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:22:23.789123 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:23.789079    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/868a944f-a78a-4d16-835b-a9fad055beb9-extensions-socket-volume\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:22:24.224611 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:24.224508    2578 generic.go:356] "Generic (PLEG): container finished" podID="868a944f-a78a-4d16-835b-a9fad055beb9" containerID="ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426" exitCode=0
Jul 07 23:22:24.224611 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:24.224554    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"
Jul 07 23:22:24.224611 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:24.224592    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc" event={"ID":"868a944f-a78a-4d16-835b-a9fad055beb9","Type":"ContainerDied","Data":"ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426"}
Jul 07 23:22:24.224858 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:24.224647    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc" event={"ID":"868a944f-a78a-4d16-835b-a9fad055beb9","Type":"ContainerDied","Data":"39bfa64dffcd397ea641272ac03ac0147fe17cf403107ffb508a92e6ab24f77d"}
Jul 07 23:22:24.224858 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:24.224672    2578 scope.go:117] "RemoveContainer" containerID="ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426"
Jul 07 23:22:24.235809 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:24.235788    2578 scope.go:117] "RemoveContainer" containerID="ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426"
Jul 07 23:22:24.236116 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:24.236095    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426\": container with ID starting with ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426 not found: ID does not exist" containerID="ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426"
Jul 07 23:22:24.236183 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:24.236130    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426"} err="failed to get container status \"ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426\": rpc error: code = NotFound desc = could not find container \"ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426\": container with ID starting with ce10673beee52604759316894a084fd9ff355ec4fcd5ffebc3729a3df52e6426 not found: ID does not exist"
Jul 07 23:22:24.247182 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:24.247151    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"]
Jul 07 23:22:24.251859 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:24.251831    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dpkpc"]
Jul 07 23:22:25.962467 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:25.962437    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="868a944f-a78a-4d16-835b-a9fad055beb9" path="/var/lib/kubelet/pods/868a944f-a78a-4d16-835b-a9fad055beb9/volumes"
Jul 07 23:22:39.635989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.635948    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"]
Jul 07 23:22:39.636498 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.636321    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="868a944f-a78a-4d16-835b-a9fad055beb9" containerName="manager"
Jul 07 23:22:39.636498 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.636331    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="868a944f-a78a-4d16-835b-a9fad055beb9" containerName="manager"
Jul 07 23:22:39.636498 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.636389    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="868a944f-a78a-4d16-835b-a9fad055beb9" containerName="manager"
Jul 07 23:22:39.640535 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.640512    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.643319 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.643295    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tqljr\""
Jul 07 23:22:39.653627 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.653593    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"]
Jul 07 23:22:39.723440 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.723407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.723440 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.723444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.723694 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.723467    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.723694 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.723486    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mkbw\" (UniqueName: \"kubernetes.io/projected/60a067ee-b2ba-469f-8b34-a6af603e638e-kube-api-access-5mkbw\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.723694 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.723616    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.723694 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.723646    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a067ee-b2ba-469f-8b34-a6af603e638e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.723694 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.723665    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.723853 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.723750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.723853 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.723803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.824921 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.824875    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825081 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.824930    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a067ee-b2ba-469f-8b34-a6af603e638e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825081 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.824950    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825185 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.825073    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825185 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.825134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825185 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.825175    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.825199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.825234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.825262    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mkbw\" (UniqueName: \"kubernetes.io/projected/60a067ee-b2ba-469f-8b34-a6af603e638e-kube-api-access-5mkbw\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825480 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.825406    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825737 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.825705    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825884 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.825780    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825884 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.825793    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a067ee-b2ba-469f-8b34-a6af603e638e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.825884 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.825723    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.827234 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.827207    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.827882 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.827863    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.840372 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.840344    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a067ee-b2ba-469f-8b34-a6af603e638e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.841473 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.841454    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mkbw\" (UniqueName: \"kubernetes.io/projected/60a067ee-b2ba-469f-8b34-a6af603e638e-kube-api-access-5mkbw\") pod \"maas-default-gateway-openshift-default-5896d4dd49-sdsln\" (UID: \"60a067ee-b2ba-469f-8b34-a6af603e638e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:39.951186 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:39.951091    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:40.142467 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:40.142444    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"]
Jul 07 23:22:40.145022 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:22:40.145000    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a067ee_b2ba_469f_8b34_a6af603e638e.slice/crio-8f6180c7a9c294e0c05ec19985060a285619c9a912475efd9cba1575fe37f5e1 WatchSource:0}: Error finding container 8f6180c7a9c294e0c05ec19985060a285619c9a912475efd9cba1575fe37f5e1: Status 404 returned error can't find the container with id 8f6180c7a9c294e0c05ec19985060a285619c9a912475efd9cba1575fe37f5e1
Jul 07 23:22:40.147147 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:40.147115    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:22:40.147220 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:40.147184    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:22:40.147220 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:40.147210    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:22:40.319596 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:40.319544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln" event={"ID":"60a067ee-b2ba-469f-8b34-a6af603e638e","Type":"ContainerStarted","Data":"b3c66a55aa612e1e89715f79f1cf501d9323d0207bb64bbae91f2529f0611758"}
Jul 07 23:22:40.319893 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:40.319873    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln" event={"ID":"60a067ee-b2ba-469f-8b34-a6af603e638e","Type":"ContainerStarted","Data":"8f6180c7a9c294e0c05ec19985060a285619c9a912475efd9cba1575fe37f5e1"}
Jul 07 23:22:40.347278 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:40.347223    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln" podStartSLOduration=1.347206542 podStartE2EDuration="1.347206542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:22:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:22:40.346155249 +0000 UTC m=+544.931724611" watchObservedRunningTime="2026-07-07 23:22:40.347206542 +0000 UTC m=+544.932775903"
Jul 07 23:22:40.951696 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:40.951645    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:41.399967 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.399926    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"]
Jul 07 23:22:41.403529 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.403509    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:41.406458 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.406429    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 07 23:22:41.406641 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.406621    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zdtnw\""
Jul 07 23:22:41.414433 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.414407    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"]
Jul 07 23:22:41.442394 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.442357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4msww\" (UniqueName: \"kubernetes.io/projected/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-kube-api-access-4msww\") pod \"limitador-limitador-64c94fd64f-bdrd6\" (UID: \"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:41.442644 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.442440    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-config-file\") pod \"limitador-limitador-64c94fd64f-bdrd6\" (UID: \"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:41.494683 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.494636    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"]
Jul 07 23:22:41.543854 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.543822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4msww\" (UniqueName: \"kubernetes.io/projected/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-kube-api-access-4msww\") pod \"limitador-limitador-64c94fd64f-bdrd6\" (UID: \"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:41.543854 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.543865    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-config-file\") pod \"limitador-limitador-64c94fd64f-bdrd6\" (UID: \"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:41.544470 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.544453    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-config-file\") pod \"limitador-limitador-64c94fd64f-bdrd6\" (UID: \"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:41.557016 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.556988    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4msww\" (UniqueName: \"kubernetes.io/projected/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-kube-api-access-4msww\") pod \"limitador-limitador-64c94fd64f-bdrd6\" (UID: \"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:41.715101 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.715010    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:41.863576 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.863533    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"]
Jul 07 23:22:41.867367 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:22:41.867338    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e3fd12d_b6f3_4912_9d54_a3fdb34522f0.slice/crio-b6605be9fa48299758e09aa21c0ee391b5e510ad0a2dd040d3ebd2ee69fa83dd WatchSource:0}: Error finding container b6605be9fa48299758e09aa21c0ee391b5e510ad0a2dd040d3ebd2ee69fa83dd: Status 404 returned error can't find the container with id b6605be9fa48299758e09aa21c0ee391b5e510ad0a2dd040d3ebd2ee69fa83dd
Jul 07 23:22:41.962476 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:41.962450    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:42.285580 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.285537    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-8c8q7"]
Jul 07 23:22:42.290310 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.290288    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8c8q7"
Jul 07 23:22:42.293461 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.293436    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4xkc5\""
Jul 07 23:22:42.297956 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.297926    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8c8q7"]
Jul 07 23:22:42.330457 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.330421    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6" event={"ID":"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0","Type":"ContainerStarted","Data":"b6605be9fa48299758e09aa21c0ee391b5e510ad0a2dd040d3ebd2ee69fa83dd"}
Jul 07 23:22:42.330689 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.330670    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:42.331622 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.331602    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:22:42.351458 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.351427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7rp7\" (UniqueName: \"kubernetes.io/projected/39a2ed31-360b-4b78-9ce2-2817e7fbc7c0-kube-api-access-n7rp7\") pod \"authorino-58cd955675-8c8q7\" (UID: \"39a2ed31-360b-4b78-9ce2-2817e7fbc7c0\") " pod="kuadrant-system/authorino-58cd955675-8c8q7"
Jul 07 23:22:42.453030 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.452985    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7rp7\" (UniqueName: \"kubernetes.io/projected/39a2ed31-360b-4b78-9ce2-2817e7fbc7c0-kube-api-access-n7rp7\") pod \"authorino-58cd955675-8c8q7\" (UID: \"39a2ed31-360b-4b78-9ce2-2817e7fbc7c0\") " pod="kuadrant-system/authorino-58cd955675-8c8q7"
Jul 07 23:22:42.464099 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.464059    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7rp7\" (UniqueName: \"kubernetes.io/projected/39a2ed31-360b-4b78-9ce2-2817e7fbc7c0-kube-api-access-n7rp7\") pod \"authorino-58cd955675-8c8q7\" (UID: \"39a2ed31-360b-4b78-9ce2-2817e7fbc7c0\") " pod="kuadrant-system/authorino-58cd955675-8c8q7"
Jul 07 23:22:42.601616 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.601494    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8c8q7"
Jul 07 23:22:42.801439 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:42.801375    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8c8q7"]
Jul 07 23:22:42.806125 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:22:42.806077    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39a2ed31_360b_4b78_9ce2_2817e7fbc7c0.slice/crio-b7a3a0c87d9e9325940a7ce4fc936621efe9aa06783879d0f5824fb4631e8a88 WatchSource:0}: Error finding container b7a3a0c87d9e9325940a7ce4fc936621efe9aa06783879d0f5824fb4631e8a88: Status 404 returned error can't find the container with id b7a3a0c87d9e9325940a7ce4fc936621efe9aa06783879d0f5824fb4631e8a88
Jul 07 23:22:43.339253 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:43.339178    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8c8q7" event={"ID":"39a2ed31-360b-4b78-9ce2-2817e7fbc7c0","Type":"ContainerStarted","Data":"b7a3a0c87d9e9325940a7ce4fc936621efe9aa06783879d0f5824fb4631e8a88"}
Jul 07 23:22:46.894587 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:46.894546    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8c8q7"]
Jul 07 23:22:47.362931 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:47.362825    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8c8q7" event={"ID":"39a2ed31-360b-4b78-9ce2-2817e7fbc7c0","Type":"ContainerStarted","Data":"541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474"}
Jul 07 23:22:47.364255 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:47.364231    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6" event={"ID":"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0","Type":"ContainerStarted","Data":"22367bf55f71a852f2f2881d63f3c113b21731d6af5fd11eb11540c3c75cc53c"}
Jul 07 23:22:47.364389 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:47.364318    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:47.382436 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:47.382381    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-8c8q7" podStartSLOduration=1.666783508 podStartE2EDuration="5.382360269s" totalImagesPullingTime="3.715576761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:22:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:22:47.38176273 +0000 UTC m=+551.967332087" watchObservedRunningTime="2026-07-07 23:22:47.382360269 +0000 UTC m=+551.967929629"
Jul 07 23:22:47.401065 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:47.401009    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6" podStartSLOduration=1.7911523 podStartE2EDuration="6.40099283s" totalImagesPullingTime="4.60984053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:22:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:22:47.399483215 +0000 UTC m=+551.985052576" watchObservedRunningTime="2026-07-07 23:22:47.40099283 +0000 UTC m=+551.986562185"
Jul 07 23:22:48.369361 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:48.369320    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-8c8q7" podUID="39a2ed31-360b-4b78-9ce2-2817e7fbc7c0" containerName="authorino" containerID="cri-o://541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474" gracePeriod=30
Jul 07 23:22:48.623010 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:48.622938    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8c8q7"
Jul 07 23:22:48.713892 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:48.713853    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7rp7\" (UniqueName: \"kubernetes.io/projected/39a2ed31-360b-4b78-9ce2-2817e7fbc7c0-kube-api-access-n7rp7\") pod \"39a2ed31-360b-4b78-9ce2-2817e7fbc7c0\" (UID: \"39a2ed31-360b-4b78-9ce2-2817e7fbc7c0\") "
Jul 07 23:22:48.716005 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:48.715970    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39a2ed31-360b-4b78-9ce2-2817e7fbc7c0-kube-api-access-n7rp7" (OuterVolumeSpecName: "kube-api-access-n7rp7") pod "39a2ed31-360b-4b78-9ce2-2817e7fbc7c0" (UID: "39a2ed31-360b-4b78-9ce2-2817e7fbc7c0"). InnerVolumeSpecName "kube-api-access-n7rp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:22:48.814654 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:48.814609    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7rp7\" (UniqueName: \"kubernetes.io/projected/39a2ed31-360b-4b78-9ce2-2817e7fbc7c0-kube-api-access-n7rp7\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:22:49.375852 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:49.375815    2578 generic.go:356] "Generic (PLEG): container finished" podID="39a2ed31-360b-4b78-9ce2-2817e7fbc7c0" containerID="541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474" exitCode=0
Jul 07 23:22:49.375852 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:49.375859    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8c8q7"
Jul 07 23:22:49.376351 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:49.375896    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8c8q7" event={"ID":"39a2ed31-360b-4b78-9ce2-2817e7fbc7c0","Type":"ContainerDied","Data":"541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474"}
Jul 07 23:22:49.376351 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:49.375941    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8c8q7" event={"ID":"39a2ed31-360b-4b78-9ce2-2817e7fbc7c0","Type":"ContainerDied","Data":"b7a3a0c87d9e9325940a7ce4fc936621efe9aa06783879d0f5824fb4631e8a88"}
Jul 07 23:22:49.376351 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:49.375958    2578 scope.go:117] "RemoveContainer" containerID="541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474"
Jul 07 23:22:49.384833 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:49.384812    2578 scope.go:117] "RemoveContainer" containerID="541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474"
Jul 07 23:22:49.385133 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:22:49.385115    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474\": container with ID starting with 541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474 not found: ID does not exist" containerID="541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474"
Jul 07 23:22:49.385186 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:49.385143    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474"} err="failed to get container status \"541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474\": rpc error: code = NotFound desc = could not find container \"541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474\": container with ID starting with 541470e6a59813d3a90d0ecc53524e9854fd8d41e9dc9ce0ad9ac23007cbf474 not found: ID does not exist"
Jul 07 23:22:49.396836 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:49.396798    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8c8q7"]
Jul 07 23:22:49.400884 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:49.400855    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-8c8q7"]
Jul 07 23:22:49.962806 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:49.962766    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39a2ed31-360b-4b78-9ce2-2817e7fbc7c0" path="/var/lib/kubelet/pods/39a2ed31-360b-4b78-9ce2-2817e7fbc7c0/volumes"
Jul 07 23:22:57.884757 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:57.884722    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"]
Jul 07 23:22:57.885145 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:57.884975    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6" podUID="2e3fd12d-b6f3-4912-9d54-a3fdb34522f0" containerName="limitador" containerID="cri-o://22367bf55f71a852f2f2881d63f3c113b21731d6af5fd11eb11540c3c75cc53c" gracePeriod=30
Jul 07 23:22:57.885661 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:57.885623    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:58.420089 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:58.420059    2578 generic.go:356] "Generic (PLEG): container finished" podID="2e3fd12d-b6f3-4912-9d54-a3fdb34522f0" containerID="22367bf55f71a852f2f2881d63f3c113b21731d6af5fd11eb11540c3c75cc53c" exitCode=0
Jul 07 23:22:58.420242 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:58.420145    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6" event={"ID":"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0","Type":"ContainerDied","Data":"22367bf55f71a852f2f2881d63f3c113b21731d6af5fd11eb11540c3c75cc53c"}
Jul 07 23:22:58.439875 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:58.439790    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:58.603329 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:58.603277    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-config-file\") pod \"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0\" (UID: \"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0\") "
Jul 07 23:22:58.603490 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:58.603387    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4msww\" (UniqueName: \"kubernetes.io/projected/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-kube-api-access-4msww\") pod \"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0\" (UID: \"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0\") "
Jul 07 23:22:58.603711 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:58.603688    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-config-file" (OuterVolumeSpecName: "config-file") pod "2e3fd12d-b6f3-4912-9d54-a3fdb34522f0" (UID: "2e3fd12d-b6f3-4912-9d54-a3fdb34522f0"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:22:58.605442 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:58.605414    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-kube-api-access-4msww" (OuterVolumeSpecName: "kube-api-access-4msww") pod "2e3fd12d-b6f3-4912-9d54-a3fdb34522f0" (UID: "2e3fd12d-b6f3-4912-9d54-a3fdb34522f0"). InnerVolumeSpecName "kube-api-access-4msww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:22:58.704334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:58.704237    2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-config-file\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:22:58.704334 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:58.704269    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4msww\" (UniqueName: \"kubernetes.io/projected/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0-kube-api-access-4msww\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:22:59.426340 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:59.426305    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"
Jul 07 23:22:59.426340 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:59.426318    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdrd6" event={"ID":"2e3fd12d-b6f3-4912-9d54-a3fdb34522f0","Type":"ContainerDied","Data":"b6605be9fa48299758e09aa21c0ee391b5e510ad0a2dd040d3ebd2ee69fa83dd"}
Jul 07 23:22:59.426867 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:59.426383    2578 scope.go:117] "RemoveContainer" containerID="22367bf55f71a852f2f2881d63f3c113b21731d6af5fd11eb11540c3c75cc53c"
Jul 07 23:22:59.449219 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:59.449180    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"]
Jul 07 23:22:59.453474 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:59.453446    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdrd6"]
Jul 07 23:22:59.962736 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:22:59.962702    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e3fd12d-b6f3-4912-9d54-a3fdb34522f0" path="/var/lib/kubelet/pods/2e3fd12d-b6f3-4912-9d54-a3fdb34522f0/volumes"
Jul 07 23:23:03.022718 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.022686    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"]
Jul 07 23:23:03.023109 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.023049    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e3fd12d-b6f3-4912-9d54-a3fdb34522f0" containerName="limitador"
Jul 07 23:23:03.023109 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.023061    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e3fd12d-b6f3-4912-9d54-a3fdb34522f0" containerName="limitador"
Jul 07 23:23:03.023109 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.023080    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a2ed31-360b-4b78-9ce2-2817e7fbc7c0" containerName="authorino"
Jul 07 23:23:03.023109 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.023087    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a2ed31-360b-4b78-9ce2-2817e7fbc7c0" containerName="authorino"
Jul 07 23:23:03.023236 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.023144    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e3fd12d-b6f3-4912-9d54-a3fdb34522f0" containerName="limitador"
Jul 07 23:23:03.023236 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.023153    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="39a2ed31-360b-4b78-9ce2-2817e7fbc7c0" containerName="authorino"
Jul 07 23:23:03.034125 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.034090    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"
Jul 07 23:23:03.038848 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.038821    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 07 23:23:03.039362 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.039335    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 07 23:23:03.039474 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.039335    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-8kmxf\""
Jul 07 23:23:03.039474 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.039415    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 07 23:23:03.040451 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.040428    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"]
Jul 07 23:23:03.144118 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.144050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-557gd\" (UniqueName: \"kubernetes.io/projected/e62ee496-b0df-4aa1-9cef-2aed0e9104eb-kube-api-access-557gd\") pod \"postgres-69fb4594cf-hw9v2\" (UID: \"e62ee496-b0df-4aa1-9cef-2aed0e9104eb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"
Jul 07 23:23:03.144298 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.144160    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e62ee496-b0df-4aa1-9cef-2aed0e9104eb-data\") pod \"postgres-69fb4594cf-hw9v2\" (UID: \"e62ee496-b0df-4aa1-9cef-2aed0e9104eb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"
Jul 07 23:23:03.245750 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.245709    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-557gd\" (UniqueName: \"kubernetes.io/projected/e62ee496-b0df-4aa1-9cef-2aed0e9104eb-kube-api-access-557gd\") pod \"postgres-69fb4594cf-hw9v2\" (UID: \"e62ee496-b0df-4aa1-9cef-2aed0e9104eb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"
Jul 07 23:23:03.245908 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.245760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e62ee496-b0df-4aa1-9cef-2aed0e9104eb-data\") pod \"postgres-69fb4594cf-hw9v2\" (UID: \"e62ee496-b0df-4aa1-9cef-2aed0e9104eb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"
Jul 07 23:23:03.246123 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.246107    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e62ee496-b0df-4aa1-9cef-2aed0e9104eb-data\") pod \"postgres-69fb4594cf-hw9v2\" (UID: \"e62ee496-b0df-4aa1-9cef-2aed0e9104eb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"
Jul 07 23:23:03.254714 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.254682    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-557gd\" (UniqueName: \"kubernetes.io/projected/e62ee496-b0df-4aa1-9cef-2aed0e9104eb-kube-api-access-557gd\") pod \"postgres-69fb4594cf-hw9v2\" (UID: \"e62ee496-b0df-4aa1-9cef-2aed0e9104eb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"
Jul 07 23:23:03.346517 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.346428    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"
Jul 07 23:23:03.481716 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:03.481692    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"]
Jul 07 23:23:03.484230 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:23:03.484200    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode62ee496_b0df_4aa1_9cef_2aed0e9104eb.slice/crio-0b07e4be5c75f4d69dc3ed02493fff734be89e9d8d96278821164149e9dfff9e WatchSource:0}: Error finding container 0b07e4be5c75f4d69dc3ed02493fff734be89e9d8d96278821164149e9dfff9e: Status 404 returned error can't find the container with id 0b07e4be5c75f4d69dc3ed02493fff734be89e9d8d96278821164149e9dfff9e
Jul 07 23:23:04.454872 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:04.454775    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2" event={"ID":"e62ee496-b0df-4aa1-9cef-2aed0e9104eb","Type":"ContainerStarted","Data":"0b07e4be5c75f4d69dc3ed02493fff734be89e9d8d96278821164149e9dfff9e"}
Jul 07 23:23:08.478875 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:08.478790    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2" event={"ID":"e62ee496-b0df-4aa1-9cef-2aed0e9104eb","Type":"ContainerStarted","Data":"de03b2fe642dd06eb6b98f6518e95254911eae172f70478862570a51115e5cbc"}
Jul 07 23:23:08.479230 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:08.478886    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"
Jul 07 23:23:08.498751 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:08.498693    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2" podStartSLOduration=0.771218922 podStartE2EDuration="5.498674828s" totalImagesPullingTime="4.727455906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:23:08.497005986 +0000 UTC m=+573.082575347" watchObservedRunningTime="2026-07-07 23:23:08.498674828 +0000 UTC m=+573.084244188"
Jul 07 23:23:14.514081 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:14.514053    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw9v2"
Jul 07 23:23:15.092384 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.092346    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bd6h4"]
Jul 07 23:23:15.112951 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.112917    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bd6h4"]
Jul 07 23:23:15.113104 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.113016    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bd6h4"
Jul 07 23:23:15.115764 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.115744    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4xkc5\""
Jul 07 23:23:15.249727 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.249683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcknf\" (UniqueName: \"kubernetes.io/projected/64f27227-f16f-41bc-948c-0a1e586ef8e6-kube-api-access-gcknf\") pod \"authorino-5d4b67c446-bd6h4\" (UID: \"64f27227-f16f-41bc-948c-0a1e586ef8e6\") " pod="kuadrant-system/authorino-5d4b67c446-bd6h4"
Jul 07 23:23:15.290075 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.290037    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bd6h4"]
Jul 07 23:23:15.290363 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:23:15.290339    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-gcknf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-bd6h4" podUID="64f27227-f16f-41bc-948c-0a1e586ef8e6"
Jul 07 23:23:15.350814 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.350717    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcknf\" (UniqueName: \"kubernetes.io/projected/64f27227-f16f-41bc-948c-0a1e586ef8e6-kube-api-access-gcknf\") pod \"authorino-5d4b67c446-bd6h4\" (UID: \"64f27227-f16f-41bc-948c-0a1e586ef8e6\") " pod="kuadrant-system/authorino-5d4b67c446-bd6h4"
Jul 07 23:23:15.362115 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.362079    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcknf\" (UniqueName: \"kubernetes.io/projected/64f27227-f16f-41bc-948c-0a1e586ef8e6-kube-api-access-gcknf\") pod \"authorino-5d4b67c446-bd6h4\" (UID: \"64f27227-f16f-41bc-948c-0a1e586ef8e6\") " pod="kuadrant-system/authorino-5d4b67c446-bd6h4"
Jul 07 23:23:15.382471 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.382436    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-677b668c87-mr9qm"]
Jul 07 23:23:15.416175 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.416139    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-677b668c87-mr9qm"]
Jul 07 23:23:15.416335 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.416262    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677b668c87-mr9qm"
Jul 07 23:23:15.419489 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.419465    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 23:23:15.512121 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.512091    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bd6h4"
Jul 07 23:23:15.529469 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.529437    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bd6h4"
Jul 07 23:23:15.553219 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.553188    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/281b4d9e-76e1-4ee5-a759-be38920aca8e-tls-cert\") pod \"authorino-677b668c87-mr9qm\" (UID: \"281b4d9e-76e1-4ee5-a759-be38920aca8e\") " pod="kuadrant-system/authorino-677b668c87-mr9qm"
Jul 07 23:23:15.553219 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.553221    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k89zm\" (UniqueName: \"kubernetes.io/projected/281b4d9e-76e1-4ee5-a759-be38920aca8e-kube-api-access-k89zm\") pod \"authorino-677b668c87-mr9qm\" (UID: \"281b4d9e-76e1-4ee5-a759-be38920aca8e\") " pod="kuadrant-system/authorino-677b668c87-mr9qm"
Jul 07 23:23:15.654230 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.654195    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcknf\" (UniqueName: \"kubernetes.io/projected/64f27227-f16f-41bc-948c-0a1e586ef8e6-kube-api-access-gcknf\") pod \"64f27227-f16f-41bc-948c-0a1e586ef8e6\" (UID: \"64f27227-f16f-41bc-948c-0a1e586ef8e6\") "
Jul 07 23:23:15.654411 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.654339    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/281b4d9e-76e1-4ee5-a759-be38920aca8e-tls-cert\") pod \"authorino-677b668c87-mr9qm\" (UID: \"281b4d9e-76e1-4ee5-a759-be38920aca8e\") " pod="kuadrant-system/authorino-677b668c87-mr9qm"
Jul 07 23:23:15.654411 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.654361    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k89zm\" (UniqueName: \"kubernetes.io/projected/281b4d9e-76e1-4ee5-a759-be38920aca8e-kube-api-access-k89zm\") pod \"authorino-677b668c87-mr9qm\" (UID: \"281b4d9e-76e1-4ee5-a759-be38920aca8e\") " pod="kuadrant-system/authorino-677b668c87-mr9qm"
Jul 07 23:23:15.656477 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.656446    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64f27227-f16f-41bc-948c-0a1e586ef8e6-kube-api-access-gcknf" (OuterVolumeSpecName: "kube-api-access-gcknf") pod "64f27227-f16f-41bc-948c-0a1e586ef8e6" (UID: "64f27227-f16f-41bc-948c-0a1e586ef8e6"). InnerVolumeSpecName "kube-api-access-gcknf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:23:15.656843 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.656822    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/281b4d9e-76e1-4ee5-a759-be38920aca8e-tls-cert\") pod \"authorino-677b668c87-mr9qm\" (UID: \"281b4d9e-76e1-4ee5-a759-be38920aca8e\") " pod="kuadrant-system/authorino-677b668c87-mr9qm"
Jul 07 23:23:15.663197 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.663178    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k89zm\" (UniqueName: \"kubernetes.io/projected/281b4d9e-76e1-4ee5-a759-be38920aca8e-kube-api-access-k89zm\") pod \"authorino-677b668c87-mr9qm\" (UID: \"281b4d9e-76e1-4ee5-a759-be38920aca8e\") " pod="kuadrant-system/authorino-677b668c87-mr9qm"
Jul 07 23:23:15.725808 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.725772    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677b668c87-mr9qm"
Jul 07 23:23:15.755285 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.755249    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcknf\" (UniqueName: \"kubernetes.io/projected/64f27227-f16f-41bc-948c-0a1e586ef8e6-kube-api-access-gcknf\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:23:15.878388 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:15.878362    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-677b668c87-mr9qm"]
Jul 07 23:23:15.880305 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:23:15.880271    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod281b4d9e_76e1_4ee5_a759_be38920aca8e.slice/crio-511d20d309ae639a8fa51dac240d4211ecb329dcbdd84823b4a4a75c92567d3f WatchSource:0}: Error finding container 511d20d309ae639a8fa51dac240d4211ecb329dcbdd84823b4a4a75c92567d3f: Status 404 returned error can't find the container with id 511d20d309ae639a8fa51dac240d4211ecb329dcbdd84823b4a4a75c92567d3f
Jul 07 23:23:16.518795 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:16.518765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-677b668c87-mr9qm" event={"ID":"281b4d9e-76e1-4ee5-a759-be38920aca8e","Type":"ContainerStarted","Data":"511d20d309ae639a8fa51dac240d4211ecb329dcbdd84823b4a4a75c92567d3f"}
Jul 07 23:23:16.518795 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:16.518787    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bd6h4"
Jul 07 23:23:16.549113 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:16.549083    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bd6h4"]
Jul 07 23:23:16.553581 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:16.553536    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bd6h4"]
Jul 07 23:23:17.525805 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:17.525764    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-677b668c87-mr9qm" event={"ID":"281b4d9e-76e1-4ee5-a759-be38920aca8e","Type":"ContainerStarted","Data":"a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc"}
Jul 07 23:23:17.543916 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:17.543858    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-677b668c87-mr9qm" podStartSLOduration=1.9776871470000001 podStartE2EDuration="2.543843658s" totalImagesPullingTime="566.156511ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:23:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:23:17.542605538 +0000 UTC m=+582.128174899" watchObservedRunningTime="2026-07-07 23:23:17.543843658 +0000 UTC m=+582.129413019"
Jul 07 23:23:17.962494 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:17.962461    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64f27227-f16f-41bc-948c-0a1e586ef8e6" path="/var/lib/kubelet/pods/64f27227-f16f-41bc-948c-0a1e586ef8e6/volumes"
Jul 07 23:23:22.220484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.220444    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85dc9d69dd-b6l86"]
Jul 07 23:23:22.224048 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.224029    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:23:22.227885 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.227860    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qcd2z\""
Jul 07 23:23:22.228003 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.227879    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 23:23:22.228003 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.227860    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 23:23:22.233768 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.233745    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85dc9d69dd-b6l86"]
Jul 07 23:23:22.315083 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.315045    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5c828d3-543b-48b6-8bc2-8ee975687e65-webhook-cert\") pod \"maas-controller-85dc9d69dd-b6l86\" (UID: \"b5c828d3-543b-48b6-8bc2-8ee975687e65\") " pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:23:22.315298 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.315122    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pld8x\" (UniqueName: \"kubernetes.io/projected/b5c828d3-543b-48b6-8bc2-8ee975687e65-kube-api-access-pld8x\") pod \"maas-controller-85dc9d69dd-b6l86\" (UID: \"b5c828d3-543b-48b6-8bc2-8ee975687e65\") " pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:23:22.416431 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.416389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pld8x\" (UniqueName: \"kubernetes.io/projected/b5c828d3-543b-48b6-8bc2-8ee975687e65-kube-api-access-pld8x\") pod \"maas-controller-85dc9d69dd-b6l86\" (UID: \"b5c828d3-543b-48b6-8bc2-8ee975687e65\") " pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:23:22.416634 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.416480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5c828d3-543b-48b6-8bc2-8ee975687e65-webhook-cert\") pod \"maas-controller-85dc9d69dd-b6l86\" (UID: \"b5c828d3-543b-48b6-8bc2-8ee975687e65\") " pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:23:22.418994 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.418969    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5c828d3-543b-48b6-8bc2-8ee975687e65-webhook-cert\") pod \"maas-controller-85dc9d69dd-b6l86\" (UID: \"b5c828d3-543b-48b6-8bc2-8ee975687e65\") " pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:23:22.426011 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.425989    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pld8x\" (UniqueName: \"kubernetes.io/projected/b5c828d3-543b-48b6-8bc2-8ee975687e65-kube-api-access-pld8x\") pod \"maas-controller-85dc9d69dd-b6l86\" (UID: \"b5c828d3-543b-48b6-8bc2-8ee975687e65\") " pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:23:22.535383 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.535281    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:23:22.675462 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:22.675431    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85dc9d69dd-b6l86"]
Jul 07 23:23:22.678133 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:23:22.678107    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c828d3_543b_48b6_8bc2_8ee975687e65.slice/crio-c94f0f8661cc14ef9ae26e8c4a5e31a0202815e66b7a114a15b48909f97fa502 WatchSource:0}: Error finding container c94f0f8661cc14ef9ae26e8c4a5e31a0202815e66b7a114a15b48909f97fa502: Status 404 returned error can't find the container with id c94f0f8661cc14ef9ae26e8c4a5e31a0202815e66b7a114a15b48909f97fa502
Jul 07 23:23:23.558917 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:23.558874    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85dc9d69dd-b6l86" event={"ID":"b5c828d3-543b-48b6-8bc2-8ee975687e65","Type":"ContainerStarted","Data":"c94f0f8661cc14ef9ae26e8c4a5e31a0202815e66b7a114a15b48909f97fa502"}
Jul 07 23:23:25.570182 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:25.570143    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85dc9d69dd-b6l86" event={"ID":"b5c828d3-543b-48b6-8bc2-8ee975687e65","Type":"ContainerStarted","Data":"910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8"}
Jul 07 23:23:25.570658 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:25.570271    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:23:25.609932 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:25.609880    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85dc9d69dd-b6l86" podStartSLOduration=1.611775616 podStartE2EDuration="3.609863381s" totalImagesPullingTime="1.998087765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:23:25.604300955 +0000 UTC m=+590.189870315" watchObservedRunningTime="2026-07-07 23:23:25.609863381 +0000 UTC m=+590.195432741"
Jul 07 23:23:36.581118 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:36.581080    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:23:56.955428 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:56.955390    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"]
Jul 07 23:23:56.983415 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:56.983366    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"]
Jul 07 23:23:56.983610 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:56.983496    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:23:56.986855 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:56.986829    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 07 23:23:56.987543 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:56.987519    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fmf5g\""
Jul 07 23:23:57.136297 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.136206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c04d0f-5f0c-47ed-8d15-c0e918489f85-maas-api-tls\") pod \"maas-api-5c6b8bff7d-kkvpg\" (UID: \"40c04d0f-5f0c-47ed-8d15-c0e918489f85\") " pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:23:57.136297 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.136270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp29j\" (UniqueName: \"kubernetes.io/projected/40c04d0f-5f0c-47ed-8d15-c0e918489f85-kube-api-access-zp29j\") pod \"maas-api-5c6b8bff7d-kkvpg\" (UID: \"40c04d0f-5f0c-47ed-8d15-c0e918489f85\") " pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:23:57.136531 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.136398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c04d0f-5f0c-47ed-8d15-c0e918489f85-openshift-service-ca\") pod \"maas-api-5c6b8bff7d-kkvpg\" (UID: \"40c04d0f-5f0c-47ed-8d15-c0e918489f85\") " pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:23:57.237752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.237657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp29j\" (UniqueName: \"kubernetes.io/projected/40c04d0f-5f0c-47ed-8d15-c0e918489f85-kube-api-access-zp29j\") pod \"maas-api-5c6b8bff7d-kkvpg\" (UID: \"40c04d0f-5f0c-47ed-8d15-c0e918489f85\") " pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:23:57.237752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.237714    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c04d0f-5f0c-47ed-8d15-c0e918489f85-openshift-service-ca\") pod \"maas-api-5c6b8bff7d-kkvpg\" (UID: \"40c04d0f-5f0c-47ed-8d15-c0e918489f85\") " pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:23:57.237990 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.237850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c04d0f-5f0c-47ed-8d15-c0e918489f85-maas-api-tls\") pod \"maas-api-5c6b8bff7d-kkvpg\" (UID: \"40c04d0f-5f0c-47ed-8d15-c0e918489f85\") " pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:23:57.238462 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.238416    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c04d0f-5f0c-47ed-8d15-c0e918489f85-openshift-service-ca\") pod \"maas-api-5c6b8bff7d-kkvpg\" (UID: \"40c04d0f-5f0c-47ed-8d15-c0e918489f85\") " pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:23:57.240375 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.240353    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c04d0f-5f0c-47ed-8d15-c0e918489f85-maas-api-tls\") pod \"maas-api-5c6b8bff7d-kkvpg\" (UID: \"40c04d0f-5f0c-47ed-8d15-c0e918489f85\") " pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:23:57.247064 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.247041    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp29j\" (UniqueName: \"kubernetes.io/projected/40c04d0f-5f0c-47ed-8d15-c0e918489f85-kube-api-access-zp29j\") pod \"maas-api-5c6b8bff7d-kkvpg\" (UID: \"40c04d0f-5f0c-47ed-8d15-c0e918489f85\") " pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:23:57.294607 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.294543    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:23:57.453798 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.453768    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"]
Jul 07 23:23:57.456180 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:23:57.456151    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c04d0f_5f0c_47ed_8d15_c0e918489f85.slice/crio-acbc6f93df110d5800e5dd8d9afc1c81842f4fd15b9bc95fc6b339f3148584e6 WatchSource:0}: Error finding container acbc6f93df110d5800e5dd8d9afc1c81842f4fd15b9bc95fc6b339f3148584e6: Status 404 returned error can't find the container with id acbc6f93df110d5800e5dd8d9afc1c81842f4fd15b9bc95fc6b339f3148584e6
Jul 07 23:23:57.501162 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.501077    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-rngq2"]
Jul 07 23:23:57.505865 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.505839    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-rngq2"
Jul 07 23:23:57.508938 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.508918    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tr9kb\""
Jul 07 23:23:57.510845 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.510821    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 07 23:23:57.520513 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.520487    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-rngq2"]
Jul 07 23:23:57.640051 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.640004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv8cv\" (UniqueName: \"kubernetes.io/projected/9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb-kube-api-access-wv8cv\") pod \"payload-processing-58f86c7798-rngq2\" (UID: \"9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb\") " pod="openshift-ingress/payload-processing-58f86c7798-rngq2"
Jul 07 23:23:57.640233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.640083    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb-plugins-config-volume\") pod \"payload-processing-58f86c7798-rngq2\" (UID: \"9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb\") " pod="openshift-ingress/payload-processing-58f86c7798-rngq2"
Jul 07 23:23:57.729271 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.729236    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg" event={"ID":"40c04d0f-5f0c-47ed-8d15-c0e918489f85","Type":"ContainerStarted","Data":"acbc6f93df110d5800e5dd8d9afc1c81842f4fd15b9bc95fc6b339f3148584e6"}
Jul 07 23:23:57.740872 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.740838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv8cv\" (UniqueName: \"kubernetes.io/projected/9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb-kube-api-access-wv8cv\") pod \"payload-processing-58f86c7798-rngq2\" (UID: \"9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb\") " pod="openshift-ingress/payload-processing-58f86c7798-rngq2"
Jul 07 23:23:57.741042 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.740890    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb-plugins-config-volume\") pod \"payload-processing-58f86c7798-rngq2\" (UID: \"9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb\") " pod="openshift-ingress/payload-processing-58f86c7798-rngq2"
Jul 07 23:23:57.741588 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.741541    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb-plugins-config-volume\") pod \"payload-processing-58f86c7798-rngq2\" (UID: \"9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb\") " pod="openshift-ingress/payload-processing-58f86c7798-rngq2"
Jul 07 23:23:57.754050 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.753977    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv8cv\" (UniqueName: \"kubernetes.io/projected/9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb-kube-api-access-wv8cv\") pod \"payload-processing-58f86c7798-rngq2\" (UID: \"9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb\") " pod="openshift-ingress/payload-processing-58f86c7798-rngq2"
Jul 07 23:23:57.818787 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.818745    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-rngq2"
Jul 07 23:23:57.990237 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:57.990203    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-rngq2"]
Jul 07 23:23:57.993641 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:23:57.993614    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb3b4fe_707e_41f6_a6a7_ded7d2c233cb.slice/crio-70a2812d6e0f9ba77f3fe91fbcd2929f9fa1c9d52906650a588d8ff0ef3aec7b WatchSource:0}: Error finding container 70a2812d6e0f9ba77f3fe91fbcd2929f9fa1c9d52906650a588d8ff0ef3aec7b: Status 404 returned error can't find the container with id 70a2812d6e0f9ba77f3fe91fbcd2929f9fa1c9d52906650a588d8ff0ef3aec7b
Jul 07 23:23:58.738614 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:23:58.738577    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-rngq2" event={"ID":"9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb","Type":"ContainerStarted","Data":"70a2812d6e0f9ba77f3fe91fbcd2929f9fa1c9d52906650a588d8ff0ef3aec7b"}
Jul 07 23:24:00.751977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:00.751934    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg" event={"ID":"40c04d0f-5f0c-47ed-8d15-c0e918489f85","Type":"ContainerStarted","Data":"a2ebd6bb280035b023381cf16ac4a4acbb35e07da07a2a3cbe6fa11865fccf6d"}
Jul 07 23:24:00.752412 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:00.752005    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:24:00.753471 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:00.753447    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-rngq2" event={"ID":"9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb","Type":"ContainerStarted","Data":"1750e9314901d0e54d17dcbf309ee9cb477eeea78515da6ba7f97cb999bcc7ac"}
Jul 07 23:24:00.753612 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:00.753492    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-rngq2"
Jul 07 23:24:00.777762 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:00.777694    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg" podStartSLOduration=2.224664686 podStartE2EDuration="4.777672321s" totalImagesPullingTime="2.553007635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:23:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:24:00.775488138 +0000 UTC m=+625.361057510" watchObservedRunningTime="2026-07-07 23:24:00.777672321 +0000 UTC m=+625.363241685"
Jul 07 23:24:00.807248 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:00.807186    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-rngq2" podStartSLOduration=1.790225071 podStartE2EDuration="3.807164346s" totalImagesPullingTime="2.016939275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:23:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:24:00.806046049 +0000 UTC m=+625.391615411" watchObservedRunningTime="2026-07-07 23:24:00.807164346 +0000 UTC m=+625.392733716"
Jul 07 23:24:05.780718 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:05.780683    2578 generic.go:356] "Generic (PLEG): container finished" podID="60a067ee-b2ba-469f-8b34-a6af603e638e" containerID="b3c66a55aa612e1e89715f79f1cf501d9323d0207bb64bbae91f2529f0611758" exitCode=0
Jul 07 23:24:05.781145 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:05.780758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln" event={"ID":"60a067ee-b2ba-469f-8b34-a6af603e638e","Type":"ContainerDied","Data":"b3c66a55aa612e1e89715f79f1cf501d9323d0207bb64bbae91f2529f0611758"}
Jul 07 23:24:05.781226 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:05.781208    2578 scope.go:117] "RemoveContainer" containerID="b3c66a55aa612e1e89715f79f1cf501d9323d0207bb64bbae91f2529f0611758"
Jul 07 23:24:05.782380 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:05.782345    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:24:05.782461 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:05.782443    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:24:05.782510 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:05.782494    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:24:06.764422 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:06.764391    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c6b8bff7d-kkvpg"
Jul 07 23:24:06.801584 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:06.799570    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln" event={"ID":"60a067ee-b2ba-469f-8b34-a6af603e638e","Type":"ContainerStarted","Data":"242e2000b4645eaa32bc2a43c8beaacae992108488a62326903ac179442f50d0"}
Jul 07 23:24:06.951460 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:06.951414    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:24:06.951776 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:06.951749    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-sdsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 07 23:24:06.951844 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:06.951817    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln" podUID="60a067ee-b2ba-469f-8b34-a6af603e638e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 07 23:24:07.952145 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:07.952107    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-sdsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 07 23:24:07.952612 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:07.952167    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln" podUID="60a067ee-b2ba-469f-8b34-a6af603e638e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 07 23:24:08.951997 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:08.951955    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-sdsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 07 23:24:08.952174 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:08.952019    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln" podUID="60a067ee-b2ba-469f-8b34-a6af603e638e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 07 23:24:09.952508 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:09.952464    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-sdsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 07 23:24:09.952980 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:09.952540    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln" podUID="60a067ee-b2ba-469f-8b34-a6af603e638e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 07 23:24:10.983205 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:10.983172    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:24:10.983704 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:10.983684    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:24:10.984631 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:10.984612    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-sdsln"
Jul 07 23:24:11.760722 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:11.760691    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-rngq2"
Jul 07 23:24:16.342135 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.342103    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"]
Jul 07 23:24:16.344748 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.344727    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.348502 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.348470    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 23:24:16.348502 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.348490    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 23:24:16.348778 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.348512    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-99wd9\""
Jul 07 23:24:16.348778 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.348470    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 07 23:24:16.359931 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.359901    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"]
Jul 07 23:24:16.405067 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.405034    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.405233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.405074    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ea3894c-863a-435d-86c1-81120d83265c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.405233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.405099    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rg4r\" (UniqueName: \"kubernetes.io/projected/4ea3894c-863a-435d-86c1-81120d83265c-kube-api-access-9rg4r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.405233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.405130    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.405233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.405216    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.405379 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.405245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.507356 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.507318    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.507676 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.507656    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.507816 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.507801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.507932 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.507919    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ea3894c-863a-435d-86c1-81120d83265c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.508046 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.508029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rg4r\" (UniqueName: \"kubernetes.io/projected/4ea3894c-863a-435d-86c1-81120d83265c-kube-api-access-9rg4r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.508153 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.508141    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.508900 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.508877    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.510633 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.510610    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.511026 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.511007    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.513439 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.513413    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ea3894c-863a-435d-86c1-81120d83265c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.514483 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.514441    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ea3894c-863a-435d-86c1-81120d83265c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.523373 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.523348    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rg4r\" (UniqueName: \"kubernetes.io/projected/4ea3894c-863a-435d-86c1-81120d83265c-kube-api-access-9rg4r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp\" (UID: \"4ea3894c-863a-435d-86c1-81120d83265c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.655700 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.655653    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:16.817715 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.817664    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"]
Jul 07 23:24:16.821145 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:24:16.821115    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea3894c_863a_435d_86c1_81120d83265c.slice/crio-417001ab4c86d636324f5966195ed3256ea36f656db1cb88c34d6b41dfc8fbf0 WatchSource:0}: Error finding container 417001ab4c86d636324f5966195ed3256ea36f656db1cb88c34d6b41dfc8fbf0: Status 404 returned error can't find the container with id 417001ab4c86d636324f5966195ed3256ea36f656db1cb88c34d6b41dfc8fbf0
Jul 07 23:24:16.822771 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.822748    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 23:24:16.849626 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:16.849590    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp" event={"ID":"4ea3894c-863a-435d-86c1-81120d83265c","Type":"ContainerStarted","Data":"417001ab4c86d636324f5966195ed3256ea36f656db1cb88c34d6b41dfc8fbf0"}
Jul 07 23:24:18.864582 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:18.864524    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp" event={"ID":"4ea3894c-863a-435d-86c1-81120d83265c","Type":"ContainerStarted","Data":"be338d0f11d9d5f23a46c20c8a4e3414cc3d1383a7bc1a0605802541dbe59e3f"}
Jul 07 23:24:18.885266 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:18.885195    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp" podStartSLOduration=1.794373417 podStartE2EDuration="2.885170523s" totalImagesPullingTime="1.090797106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:24:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:24:18.88237202 +0000 UTC m=+643.467941404" watchObservedRunningTime="2026-07-07 23:24:18.885170523 +0000 UTC m=+643.470739888"
Jul 07 23:24:26.656327 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:26.656282    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:26.656771 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:26.656375    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:26.669080 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:26.669048    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:26.917834 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:26.917741    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp"
Jul 07 23:24:44.190367 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.190330    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"]
Jul 07 23:24:44.195914 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.195890    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.204691 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.204661    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 07 23:24:44.217869 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.217839    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"]
Jul 07 23:24:44.359431 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.359394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.359673 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.359441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.359673 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.359461    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fkk6\" (UniqueName: \"kubernetes.io/projected/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-kube-api-access-2fkk6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.359673 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.359592    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.359673 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.359637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.359673 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.359661    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.460819 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.460715    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.460819 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.460762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.460819 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.460787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.461167 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.460880    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.461167 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.460921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.461167 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.460944    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fkk6\" (UniqueName: \"kubernetes.io/projected/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-kube-api-access-2fkk6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.461320 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.461242    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.461320 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.461247    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.461406 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.461364    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.463179 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.463158    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.463432 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.463407    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.474948 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.474919    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fkk6\" (UniqueName: \"kubernetes.io/projected/b61808c5-1ac4-465b-8ae9-0ec8d3485cff-kube-api-access-2fkk6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9qsxw\" (UID: \"b61808c5-1ac4-465b-8ae9-0ec8d3485cff\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.507360 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.507321    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:44.648875 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.648847    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"]
Jul 07 23:24:44.650451 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:24:44.650425    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb61808c5_1ac4_465b_8ae9_0ec8d3485cff.slice/crio-d6f1d04b928a8aabca2cdc9d53042cf9194b7e05d649d222278036a2a9b7e0db WatchSource:0}: Error finding container d6f1d04b928a8aabca2cdc9d53042cf9194b7e05d649d222278036a2a9b7e0db: Status 404 returned error can't find the container with id d6f1d04b928a8aabca2cdc9d53042cf9194b7e05d649d222278036a2a9b7e0db
Jul 07 23:24:44.998648 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.998608    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw" event={"ID":"b61808c5-1ac4-465b-8ae9-0ec8d3485cff","Type":"ContainerStarted","Data":"7b3b1254197b841b317a4de75e05449f5b4bf951f5d83725e72aedb61226a805"}
Jul 07 23:24:44.998826 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:44.998664    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw" event={"ID":"b61808c5-1ac4-465b-8ae9-0ec8d3485cff","Type":"ContainerStarted","Data":"d6f1d04b928a8aabca2cdc9d53042cf9194b7e05d649d222278036a2a9b7e0db"}
Jul 07 23:24:45.019503 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:45.019445    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw" podStartSLOduration=0.851828462 podStartE2EDuration="1.01942865s" totalImagesPullingTime="167.600188ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:24:45.016696831 +0000 UTC m=+669.602266219" watchObservedRunningTime="2026-07-07 23:24:45.01942865 +0000 UTC m=+669.604998010"
Jul 07 23:24:54.507811 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:54.507770    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:54.508363 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:54.507847    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:54.520449 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:54.520423    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:24:55.058786 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:24:55.058755    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9qsxw"
Jul 07 23:26:14.962167 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:14.962079    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-677b668c87-mr9qm"]
Jul 07 23:26:14.962656 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:14.962250    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-677b668c87-mr9qm" podUID="281b4d9e-76e1-4ee5-a759-be38920aca8e" containerName="authorino" containerID="cri-o://a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc" gracePeriod=30
Jul 07 23:26:15.222099 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.222023    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677b668c87-mr9qm"
Jul 07 23:26:15.302870 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.302831    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/281b4d9e-76e1-4ee5-a759-be38920aca8e-tls-cert\") pod \"281b4d9e-76e1-4ee5-a759-be38920aca8e\" (UID: \"281b4d9e-76e1-4ee5-a759-be38920aca8e\") "
Jul 07 23:26:15.303049 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.302938    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k89zm\" (UniqueName: \"kubernetes.io/projected/281b4d9e-76e1-4ee5-a759-be38920aca8e-kube-api-access-k89zm\") pod \"281b4d9e-76e1-4ee5-a759-be38920aca8e\" (UID: \"281b4d9e-76e1-4ee5-a759-be38920aca8e\") "
Jul 07 23:26:15.305017 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.304975    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/281b4d9e-76e1-4ee5-a759-be38920aca8e-kube-api-access-k89zm" (OuterVolumeSpecName: "kube-api-access-k89zm") pod "281b4d9e-76e1-4ee5-a759-be38920aca8e" (UID: "281b4d9e-76e1-4ee5-a759-be38920aca8e"). InnerVolumeSpecName "kube-api-access-k89zm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:26:15.315776 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.315738    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/281b4d9e-76e1-4ee5-a759-be38920aca8e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "281b4d9e-76e1-4ee5-a759-be38920aca8e" (UID: "281b4d9e-76e1-4ee5-a759-be38920aca8e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:26:15.404326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.404286    2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/281b4d9e-76e1-4ee5-a759-be38920aca8e-tls-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:26:15.404326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.404318    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k89zm\" (UniqueName: \"kubernetes.io/projected/281b4d9e-76e1-4ee5-a759-be38920aca8e-kube-api-access-k89zm\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:26:15.447043 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.447005    2578 generic.go:356] "Generic (PLEG): container finished" podID="281b4d9e-76e1-4ee5-a759-be38920aca8e" containerID="a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc" exitCode=0
Jul 07 23:26:15.447214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.447045    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-677b668c87-mr9qm" event={"ID":"281b4d9e-76e1-4ee5-a759-be38920aca8e","Type":"ContainerDied","Data":"a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc"}
Jul 07 23:26:15.447214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.447065    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677b668c87-mr9qm"
Jul 07 23:26:15.447214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.447100    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-677b668c87-mr9qm" event={"ID":"281b4d9e-76e1-4ee5-a759-be38920aca8e","Type":"ContainerDied","Data":"511d20d309ae639a8fa51dac240d4211ecb329dcbdd84823b4a4a75c92567d3f"}
Jul 07 23:26:15.447214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.447122    2578 scope.go:117] "RemoveContainer" containerID="a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc"
Jul 07 23:26:15.456821 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.456797    2578 scope.go:117] "RemoveContainer" containerID="a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc"
Jul 07 23:26:15.457129 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:26:15.457110    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc\": container with ID starting with a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc not found: ID does not exist" containerID="a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc"
Jul 07 23:26:15.457193 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.457140    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc"} err="failed to get container status \"a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc\": rpc error: code = NotFound desc = could not find container \"a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc\": container with ID starting with a0e49aee3b9855e804b6e8fba28fdc07d5b6c5f37071f0f4e751cbd2a53ed1dc not found: ID does not exist"
Jul 07 23:26:15.490547 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.490453    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-677b668c87-mr9qm"]
Jul 07 23:26:15.507607 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.507573    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-677b668c87-mr9qm"]
Jul 07 23:26:15.963582 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:26:15.963530    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="281b4d9e-76e1-4ee5-a759-be38920aca8e" path="/var/lib/kubelet/pods/281b4d9e-76e1-4ee5-a759-be38920aca8e/volumes"
Jul 07 23:27:25.349157 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.349095    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85dc9d69dd-b6l86"]
Jul 07 23:27:25.349834 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.349582    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-85dc9d69dd-b6l86" podUID="b5c828d3-543b-48b6-8bc2-8ee975687e65" containerName="manager" containerID="cri-o://910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8" gracePeriod=30
Jul 07 23:27:25.609857 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.609779    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:27:25.632693 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.632659    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pld8x\" (UniqueName: \"kubernetes.io/projected/b5c828d3-543b-48b6-8bc2-8ee975687e65-kube-api-access-pld8x\") pod \"b5c828d3-543b-48b6-8bc2-8ee975687e65\" (UID: \"b5c828d3-543b-48b6-8bc2-8ee975687e65\") "
Jul 07 23:27:25.632693 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.632703    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5c828d3-543b-48b6-8bc2-8ee975687e65-webhook-cert\") pod \"b5c828d3-543b-48b6-8bc2-8ee975687e65\" (UID: \"b5c828d3-543b-48b6-8bc2-8ee975687e65\") "
Jul 07 23:27:25.635071 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.635040    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5c828d3-543b-48b6-8bc2-8ee975687e65-kube-api-access-pld8x" (OuterVolumeSpecName: "kube-api-access-pld8x") pod "b5c828d3-543b-48b6-8bc2-8ee975687e65" (UID: "b5c828d3-543b-48b6-8bc2-8ee975687e65"). InnerVolumeSpecName "kube-api-access-pld8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:27:25.635198 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.635033    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5c828d3-543b-48b6-8bc2-8ee975687e65-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b5c828d3-543b-48b6-8bc2-8ee975687e65" (UID: "b5c828d3-543b-48b6-8bc2-8ee975687e65"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:27:25.733435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.733386    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pld8x\" (UniqueName: \"kubernetes.io/projected/b5c828d3-543b-48b6-8bc2-8ee975687e65-kube-api-access-pld8x\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:27:25.733435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.733425    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5c828d3-543b-48b6-8bc2-8ee975687e65-webhook-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:27:25.788664 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.788626    2578 generic.go:356] "Generic (PLEG): container finished" podID="b5c828d3-543b-48b6-8bc2-8ee975687e65" containerID="910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8" exitCode=0
Jul 07 23:27:25.788864 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.788670    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85dc9d69dd-b6l86"
Jul 07 23:27:25.788864 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.788712    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85dc9d69dd-b6l86" event={"ID":"b5c828d3-543b-48b6-8bc2-8ee975687e65","Type":"ContainerDied","Data":"910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8"}
Jul 07 23:27:25.788864 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.788774    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85dc9d69dd-b6l86" event={"ID":"b5c828d3-543b-48b6-8bc2-8ee975687e65","Type":"ContainerDied","Data":"c94f0f8661cc14ef9ae26e8c4a5e31a0202815e66b7a114a15b48909f97fa502"}
Jul 07 23:27:25.788864 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.788806    2578 scope.go:117] "RemoveContainer" containerID="910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8"
Jul 07 23:27:25.798159 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.798141    2578 scope.go:117] "RemoveContainer" containerID="910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8"
Jul 07 23:27:25.798454 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:27:25.798434    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8\": container with ID starting with 910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8 not found: ID does not exist" containerID="910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8"
Jul 07 23:27:25.798519 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.798468    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8"} err="failed to get container status \"910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8\": rpc error: code = NotFound desc = could not find container \"910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8\": container with ID starting with 910c338f83c0c801bebd8c039b613db101faa97f78f1399e390c22788cc89ed8 not found: ID does not exist"
Jul 07 23:27:25.812223 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.812191    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85dc9d69dd-b6l86"]
Jul 07 23:27:25.816104 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.816074    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-85dc9d69dd-b6l86"]
Jul 07 23:27:25.962748 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:25.962717    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5c828d3-543b-48b6-8bc2-8ee975687e65" path="/var/lib/kubelet/pods/b5c828d3-543b-48b6-8bc2-8ee975687e65/volumes"
Jul 07 23:27:26.600304 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.600267    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85dc9d69dd-6ts5f"]
Jul 07 23:27:26.600729 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.600689    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="281b4d9e-76e1-4ee5-a759-be38920aca8e" containerName="authorino"
Jul 07 23:27:26.600729 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.600704    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="281b4d9e-76e1-4ee5-a759-be38920aca8e" containerName="authorino"
Jul 07 23:27:26.600809 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.600739    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5c828d3-543b-48b6-8bc2-8ee975687e65" containerName="manager"
Jul 07 23:27:26.600809 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.600746    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5c828d3-543b-48b6-8bc2-8ee975687e65" containerName="manager"
Jul 07 23:27:26.600884 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.600815    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="281b4d9e-76e1-4ee5-a759-be38920aca8e" containerName="authorino"
Jul 07 23:27:26.600884 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.600824    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5c828d3-543b-48b6-8bc2-8ee975687e65" containerName="manager"
Jul 07 23:27:26.605581 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.605543    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85dc9d69dd-6ts5f"
Jul 07 23:27:26.608337 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.608312    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 23:27:26.609472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.609448    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qcd2z\""
Jul 07 23:27:26.609472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.609471    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 23:27:26.613408 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.613059    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85dc9d69dd-6ts5f"]
Jul 07 23:27:26.640713 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.640678    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecad7d8d-213a-485b-95f8-b0b809d04dce-webhook-cert\") pod \"maas-controller-85dc9d69dd-6ts5f\" (UID: \"ecad7d8d-213a-485b-95f8-b0b809d04dce\") " pod="opendatahub/maas-controller-85dc9d69dd-6ts5f"
Jul 07 23:27:26.640901 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.640720    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7fnm\" (UniqueName: \"kubernetes.io/projected/ecad7d8d-213a-485b-95f8-b0b809d04dce-kube-api-access-z7fnm\") pod \"maas-controller-85dc9d69dd-6ts5f\" (UID: \"ecad7d8d-213a-485b-95f8-b0b809d04dce\") " pod="opendatahub/maas-controller-85dc9d69dd-6ts5f"
Jul 07 23:27:26.741748 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.741709    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecad7d8d-213a-485b-95f8-b0b809d04dce-webhook-cert\") pod \"maas-controller-85dc9d69dd-6ts5f\" (UID: \"ecad7d8d-213a-485b-95f8-b0b809d04dce\") " pod="opendatahub/maas-controller-85dc9d69dd-6ts5f"
Jul 07 23:27:26.741952 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.741758    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7fnm\" (UniqueName: \"kubernetes.io/projected/ecad7d8d-213a-485b-95f8-b0b809d04dce-kube-api-access-z7fnm\") pod \"maas-controller-85dc9d69dd-6ts5f\" (UID: \"ecad7d8d-213a-485b-95f8-b0b809d04dce\") " pod="opendatahub/maas-controller-85dc9d69dd-6ts5f"
Jul 07 23:27:26.744380 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.744344    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecad7d8d-213a-485b-95f8-b0b809d04dce-webhook-cert\") pod \"maas-controller-85dc9d69dd-6ts5f\" (UID: \"ecad7d8d-213a-485b-95f8-b0b809d04dce\") " pod="opendatahub/maas-controller-85dc9d69dd-6ts5f"
Jul 07 23:27:26.751266 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.751234    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7fnm\" (UniqueName: \"kubernetes.io/projected/ecad7d8d-213a-485b-95f8-b0b809d04dce-kube-api-access-z7fnm\") pod \"maas-controller-85dc9d69dd-6ts5f\" (UID: \"ecad7d8d-213a-485b-95f8-b0b809d04dce\") " pod="opendatahub/maas-controller-85dc9d69dd-6ts5f"
Jul 07 23:27:26.917250 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:26.917213    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85dc9d69dd-6ts5f"
Jul 07 23:27:27.058272 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:27.058242    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85dc9d69dd-6ts5f"]
Jul 07 23:27:27.060414 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:27:27.060382    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecad7d8d_213a_485b_95f8_b0b809d04dce.slice/crio-2599bcb1274d046c28169d3056bfa89e0b5680bb03abf59ee41c590e0d0d5fba WatchSource:0}: Error finding container 2599bcb1274d046c28169d3056bfa89e0b5680bb03abf59ee41c590e0d0d5fba: Status 404 returned error can't find the container with id 2599bcb1274d046c28169d3056bfa89e0b5680bb03abf59ee41c590e0d0d5fba
Jul 07 23:27:27.802314 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:27.802222    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85dc9d69dd-6ts5f" event={"ID":"ecad7d8d-213a-485b-95f8-b0b809d04dce","Type":"ContainerStarted","Data":"9501ed1bd085808864a1e7bcae93d1d68740a17a5f3635c9d0ff7f8a12b76767"}
Jul 07 23:27:27.802314 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:27.802275    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85dc9d69dd-6ts5f" event={"ID":"ecad7d8d-213a-485b-95f8-b0b809d04dce","Type":"ContainerStarted","Data":"2599bcb1274d046c28169d3056bfa89e0b5680bb03abf59ee41c590e0d0d5fba"}
Jul 07 23:27:27.802778 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:27.802310    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85dc9d69dd-6ts5f"
Jul 07 23:27:27.821355 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:27.821288    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85dc9d69dd-6ts5f" podStartSLOduration=1.3305097670000001 podStartE2EDuration="1.821269236s" totalImagesPullingTime="490.759469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:27:27.820138713 +0000 UTC m=+832.405708104" watchObservedRunningTime="2026-07-07 23:27:27.821269236 +0000 UTC m=+832.406838597"
Jul 07 23:27:38.812575 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:27:38.812522    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85dc9d69dd-6ts5f"
Jul 07 23:30:00.144951 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:30:00.144859    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"]
Jul 07 23:30:00.148442 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:30:00.148417    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"
Jul 07 23:30:00.163793 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:30:00.163761    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"]
Jul 07 23:30:00.244134 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:30:00.244089    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwv27\" (UniqueName: \"kubernetes.io/projected/b0965e45-f49e-42b4-9540-d1dc963626fb-kube-api-access-kwv27\") pod \"maas-api-key-cleanup-29724450-pgs79\" (UID: \"b0965e45-f49e-42b4-9540-d1dc963626fb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"
Jul 07 23:30:00.345100 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:30:00.345052    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwv27\" (UniqueName: \"kubernetes.io/projected/b0965e45-f49e-42b4-9540-d1dc963626fb-kube-api-access-kwv27\") pod \"maas-api-key-cleanup-29724450-pgs79\" (UID: \"b0965e45-f49e-42b4-9540-d1dc963626fb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"
Jul 07 23:30:00.354449 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:30:00.354420    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwv27\" (UniqueName: \"kubernetes.io/projected/b0965e45-f49e-42b4-9540-d1dc963626fb-kube-api-access-kwv27\") pod \"maas-api-key-cleanup-29724450-pgs79\" (UID: \"b0965e45-f49e-42b4-9540-d1dc963626fb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"
Jul 07 23:30:00.458904 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:30:00.458804    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"
Jul 07 23:30:00.599844 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:30:00.599813    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"]
Jul 07 23:30:00.602153 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:30:00.602120    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0965e45_f49e_42b4_9540_d1dc963626fb.slice/crio-75cc0391f506bcce8f026f5ffe254df25841629488cce663af4260a80af00ba6 WatchSource:0}: Error finding container 75cc0391f506bcce8f026f5ffe254df25841629488cce663af4260a80af00ba6: Status 404 returned error can't find the container with id 75cc0391f506bcce8f026f5ffe254df25841629488cce663af4260a80af00ba6
Jul 07 23:30:00.604211 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:30:00.604189    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 23:30:01.558425 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:30:01.558384    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79" event={"ID":"b0965e45-f49e-42b4-9540-d1dc963626fb","Type":"ContainerStarted","Data":"75cc0391f506bcce8f026f5ffe254df25841629488cce663af4260a80af00ba6"}
Jul 07 23:30:03.571035 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:30:03.570932    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79" event={"ID":"b0965e45-f49e-42b4-9540-d1dc963626fb","Type":"ContainerStarted","Data":"ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f"}
Jul 07 23:31:00.021698 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:00.021635    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79" podStartSLOduration=57.594364048 podStartE2EDuration="1m0.02161719s" totalImagesPullingTime="2.427253142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:30:03.595788016 +0000 UTC m=+988.181357392" watchObservedRunningTime="2026-07-07 23:31:00.02161719 +0000 UTC m=+1044.607186550"
Jul 07 23:31:00.022220 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:00.021820    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"]
Jul 07 23:31:00.022220 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:00.022001    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79" podUID="b0965e45-f49e-42b4-9540-d1dc963626fb" containerName="cleanup" containerID="cri-o://ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f" gracePeriod=30
Jul 07 23:31:30.221681 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:30.221650    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"
Jul 07 23:31:30.359993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:30.359898    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwv27\" (UniqueName: \"kubernetes.io/projected/b0965e45-f49e-42b4-9540-d1dc963626fb-kube-api-access-kwv27\") pod \"b0965e45-f49e-42b4-9540-d1dc963626fb\" (UID: \"b0965e45-f49e-42b4-9540-d1dc963626fb\") "
Jul 07 23:31:30.362077 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:30.362042    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0965e45-f49e-42b4-9540-d1dc963626fb-kube-api-access-kwv27" (OuterVolumeSpecName: "kube-api-access-kwv27") pod "b0965e45-f49e-42b4-9540-d1dc963626fb" (UID: "b0965e45-f49e-42b4-9540-d1dc963626fb"). InnerVolumeSpecName "kube-api-access-kwv27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:31:30.460896 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:30.460854    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwv27\" (UniqueName: \"kubernetes.io/projected/b0965e45-f49e-42b4-9540-d1dc963626fb-kube-api-access-kwv27\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:31:31.003906 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:31.003868    2578 generic.go:356] "Generic (PLEG): container finished" podID="b0965e45-f49e-42b4-9540-d1dc963626fb" containerID="ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f" exitCode=137
Jul 07 23:31:31.004127 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:31.003916    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"
Jul 07 23:31:31.004127 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:31.003961    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79" event={"ID":"b0965e45-f49e-42b4-9540-d1dc963626fb","Type":"ContainerDied","Data":"ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f"}
Jul 07 23:31:31.004127 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:31.004023    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79" event={"ID":"b0965e45-f49e-42b4-9540-d1dc963626fb","Type":"ContainerDied","Data":"75cc0391f506bcce8f026f5ffe254df25841629488cce663af4260a80af00ba6"}
Jul 07 23:31:31.004127 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:31.004052    2578 scope.go:117] "RemoveContainer" containerID="ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f"
Jul 07 23:31:31.012825 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:31.012805    2578 scope.go:117] "RemoveContainer" containerID="ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f"
Jul 07 23:31:31.013168 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:31:31.013142    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f\": container with ID starting with ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f not found: ID does not exist" containerID="ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f"
Jul 07 23:31:31.013263 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:31.013175    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f"} err="failed to get container status \"ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f\": rpc error: code = NotFound desc = could not find container \"ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f\": container with ID starting with ec9898280d96d3ab559afc4d143ee2720aed0bedb7efd2387d33a6d128877d8f not found: ID does not exist"
Jul 07 23:31:31.027819 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:31.027786    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"]
Jul 07 23:31:31.040351 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:31.040313    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724450-pgs79"]
Jul 07 23:31:31.963580 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:31:31.963532    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0965e45-f49e-42b4-9540-d1dc963626fb" path="/var/lib/kubelet/pods/b0965e45-f49e-42b4-9540-d1dc963626fb/volumes"
Jul 07 23:39:36.855435 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.855394    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"]
Jul 07 23:39:36.857909 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.855842    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0965e45-f49e-42b4-9540-d1dc963626fb" containerName="cleanup"
Jul 07 23:39:36.857909 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.855855    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0965e45-f49e-42b4-9540-d1dc963626fb" containerName="cleanup"
Jul 07 23:39:36.857909 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.855939    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0965e45-f49e-42b4-9540-d1dc963626fb" containerName="cleanup"
Jul 07 23:39:36.859264 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.859238    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:36.862325 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.862295    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f10e4379-openshift-default-dockercfg-hddjc\""
Jul 07 23:39:36.882116 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.882076    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"]
Jul 07 23:39:36.927949 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.927913    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-data\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:36.927949 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.927953    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/493b17a7-e438-41e7-9255-f20718b83336-istiod-ca-cert\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:36.928176 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.927978    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-envoy\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:36.928176 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.927993    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzxq5\" (UniqueName: \"kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-kube-api-access-pzxq5\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:36.928176 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.928028    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-certs\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:36.928176 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.928046    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-istio-token\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:36.928176 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.928109    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/493b17a7-e438-41e7-9255-f20718b83336-istio-podinfo\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:36.928176 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.928140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-socket\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:36.928398 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:36.928183    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-credential-socket\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.028633 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.028591    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-certs\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.028808 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.028652    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-istio-token\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.028808 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.028692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/493b17a7-e438-41e7-9255-f20718b83336-istio-podinfo\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.028808 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.028723    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-socket\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.028808 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.028774    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-credential-socket\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.028989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.028827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-data\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.028989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.028865    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/493b17a7-e438-41e7-9255-f20718b83336-istiod-ca-cert\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.028989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.028911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-envoy\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.028989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.028936    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzxq5\" (UniqueName: \"kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-kube-api-access-pzxq5\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.029265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.029048    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-certs\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.029265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.029159    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-socket\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.029390 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.029360    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-data\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.029686 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.029657    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-credential-socket\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.029811 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.029711    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/493b17a7-e438-41e7-9255-f20718b83336-istiod-ca-cert\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.031268 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.031241    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-envoy\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.031268 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.031254    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/493b17a7-e438-41e7-9255-f20718b83336-istio-podinfo\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.037053 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.037024    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-istio-token\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.037188 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.037169    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzxq5\" (UniqueName: \"kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-kube-api-access-pzxq5\") pod \"e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") " pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.170913 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.170878    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:37.314712 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.314687    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"]
Jul 07 23:39:37.317067 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:39:37.317022    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493b17a7_e438_41e7_9255_f20718b83336.slice/crio-a2c375860a773508316b9a2c836a45435ae3efe5780bea527316bb63f2fe03ee WatchSource:0}: Error finding container a2c375860a773508316b9a2c836a45435ae3efe5780bea527316bb63f2fe03ee: Status 404 returned error can't find the container with id a2c375860a773508316b9a2c836a45435ae3efe5780bea527316bb63f2fe03ee
Jul 07 23:39:37.319362 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.319332    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:39:37.319452 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.319398    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:39:37.319452 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.319425    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:39:37.367587 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:37.367539    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6" event={"ID":"493b17a7-e438-41e7-9255-f20718b83336","Type":"ContainerStarted","Data":"a2c375860a773508316b9a2c836a45435ae3efe5780bea527316bb63f2fe03ee"}
Jul 07 23:39:38.144481 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.144443    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"]
Jul 07 23:39:38.147983 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.147962    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:38.150833 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.150813    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f10e4379-serving-cert\""
Jul 07 23:39:38.174026 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.173994    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"]
Jul 07 23:39:38.239203 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.239151    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74f4w\" (UniqueName: \"kubernetes.io/projected/b186e191-6492-4181-ab75-97633492a077-kube-api-access-74f4w\") pod \"maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:38.239399 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.239211    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b186e191-6492-4181-ab75-97633492a077-openshift-service-ca\") pod \"maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:38.239399 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.239259    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b186e191-6492-4181-ab75-97633492a077-maas-api-tls\") pod \"maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:38.339890 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.339853    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74f4w\" (UniqueName: \"kubernetes.io/projected/b186e191-6492-4181-ab75-97633492a077-kube-api-access-74f4w\") pod \"maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:38.340095 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.339899    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b186e191-6492-4181-ab75-97633492a077-openshift-service-ca\") pod \"maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:38.340095 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.339945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b186e191-6492-4181-ab75-97633492a077-maas-api-tls\") pod \"maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:38.340095 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:39:38.340072    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-serving-cert: secret "maas-api-e2e-ait-f10e4379-serving-cert" not found
Jul 07 23:39:38.340255 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:39:38.340139    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b186e191-6492-4181-ab75-97633492a077-maas-api-tls podName:b186e191-6492-4181-ab75-97633492a077 nodeName:}" failed. No retries permitted until 2026-07-07 23:39:38.840119245 +0000 UTC m=+1563.425688585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b186e191-6492-4181-ab75-97633492a077-maas-api-tls") pod "maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2" (UID: "b186e191-6492-4181-ab75-97633492a077") : secret "maas-api-e2e-ait-f10e4379-serving-cert" not found
Jul 07 23:39:38.340752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.340729    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b186e191-6492-4181-ab75-97633492a077-openshift-service-ca\") pod \"maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:38.351615 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.351587    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74f4w\" (UniqueName: \"kubernetes.io/projected/b186e191-6492-4181-ab75-97633492a077-kube-api-access-74f4w\") pod \"maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:38.374721 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.374688    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6" event={"ID":"493b17a7-e438-41e7-9255-f20718b83336","Type":"ContainerStarted","Data":"955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5"}
Jul 07 23:39:38.404514 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.404404    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6" podStartSLOduration=2.404385761 podStartE2EDuration="2.404385761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:39:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:39:38.401874531 +0000 UTC m=+1562.987443892" watchObservedRunningTime="2026-07-07 23:39:38.404385761 +0000 UTC m=+1562.989955417"
Jul 07 23:39:38.843910 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.843790    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b186e191-6492-4181-ab75-97633492a077-maas-api-tls\") pod \"maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:38.846355 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:38.846328    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b186e191-6492-4181-ab75-97633492a077-maas-api-tls\") pod \"maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:39.059262 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:39.059228    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:39.171930 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:39.171893    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:39.177867 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:39.177840    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:39.219281 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:39.219253    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"]
Jul 07 23:39:39.221542 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:39:39.221518    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb186e191_6492_4181_ab75_97633492a077.slice/crio-389e2c83ad2d2831cbb3e01d8d82c13dd46a337561947411b2c1ae0faf74e1be WatchSource:0}: Error finding container 389e2c83ad2d2831cbb3e01d8d82c13dd46a337561947411b2c1ae0faf74e1be: Status 404 returned error can't find the container with id 389e2c83ad2d2831cbb3e01d8d82c13dd46a337561947411b2c1ae0faf74e1be
Jul 07 23:39:39.223333 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:39.223315    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 23:39:39.381411 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:39.381377    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2" event={"ID":"b186e191-6492-4181-ab75-97633492a077","Type":"ContainerStarted","Data":"389e2c83ad2d2831cbb3e01d8d82c13dd46a337561947411b2c1ae0faf74e1be"}
Jul 07 23:39:39.381710 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:39.381693    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:39.382592 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:39.382555    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:40.387553 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:40.387517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2" event={"ID":"b186e191-6492-4181-ab75-97633492a077","Type":"ContainerStarted","Data":"2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d"}
Jul 07 23:39:40.388142 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:40.387817    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:40.407101 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:40.407049    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2" podStartSLOduration=1.969455261 podStartE2EDuration="2.407033184s" totalImagesPullingTime="437.577923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:39:40.405394968 +0000 UTC m=+1564.990964328" watchObservedRunningTime="2026-07-07 23:39:40.407033184 +0000 UTC m=+1564.992602544"
Jul 07 23:39:43.231136 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.231099    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"]
Jul 07 23:39:43.231590 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.231303    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6" podUID="493b17a7-e438-41e7-9255-f20718b83336" containerName="istio-proxy" containerID="cri-o://955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5" gracePeriod=30
Jul 07 23:39:43.424360 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.424324    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"]
Jul 07 23:39:43.424645 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.424615    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2" podUID="b186e191-6492-4181-ab75-97633492a077" containerName="maas-api" containerID="cri-o://2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d" gracePeriod=30
Jul 07 23:39:43.722461 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.722437    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:43.804706 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.804613    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b186e191-6492-4181-ab75-97633492a077-maas-api-tls\") pod \"b186e191-6492-4181-ab75-97633492a077\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") "
Jul 07 23:39:43.804706 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.804703    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74f4w\" (UniqueName: \"kubernetes.io/projected/b186e191-6492-4181-ab75-97633492a077-kube-api-access-74f4w\") pod \"b186e191-6492-4181-ab75-97633492a077\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") "
Jul 07 23:39:43.804944 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.804769    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b186e191-6492-4181-ab75-97633492a077-openshift-service-ca\") pod \"b186e191-6492-4181-ab75-97633492a077\" (UID: \"b186e191-6492-4181-ab75-97633492a077\") "
Jul 07 23:39:43.805148 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.805122    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b186e191-6492-4181-ab75-97633492a077-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b186e191-6492-4181-ab75-97633492a077" (UID: "b186e191-6492-4181-ab75-97633492a077"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:39:43.807271 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.807245    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b186e191-6492-4181-ab75-97633492a077-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b186e191-6492-4181-ab75-97633492a077" (UID: "b186e191-6492-4181-ab75-97633492a077"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:39:43.807386 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.807364    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b186e191-6492-4181-ab75-97633492a077-kube-api-access-74f4w" (OuterVolumeSpecName: "kube-api-access-74f4w") pod "b186e191-6492-4181-ab75-97633492a077" (UID: "b186e191-6492-4181-ab75-97633492a077"). InnerVolumeSpecName "kube-api-access-74f4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:39:43.905536 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.905499    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74f4w\" (UniqueName: \"kubernetes.io/projected/b186e191-6492-4181-ab75-97633492a077-kube-api-access-74f4w\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:43.905536 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.905531    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b186e191-6492-4181-ab75-97633492a077-openshift-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:43.905536 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:43.905545    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b186e191-6492-4181-ab75-97633492a077-maas-api-tls\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:44.411815 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:44.411778    2578 generic.go:356] "Generic (PLEG): container finished" podID="b186e191-6492-4181-ab75-97633492a077" containerID="2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d" exitCode=0
Jul 07 23:39:44.412216 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:44.411824    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"
Jul 07 23:39:44.412216 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:44.411867    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2" event={"ID":"b186e191-6492-4181-ab75-97633492a077","Type":"ContainerDied","Data":"2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d"}
Jul 07 23:39:44.412216 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:44.411918    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2" event={"ID":"b186e191-6492-4181-ab75-97633492a077","Type":"ContainerDied","Data":"389e2c83ad2d2831cbb3e01d8d82c13dd46a337561947411b2c1ae0faf74e1be"}
Jul 07 23:39:44.412216 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:44.411941    2578 scope.go:117] "RemoveContainer" containerID="2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d"
Jul 07 23:39:44.420973 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:44.420956    2578 scope.go:117] "RemoveContainer" containerID="2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d"
Jul 07 23:39:44.421278 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:39:44.421254    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d\": container with ID starting with 2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d not found: ID does not exist" containerID="2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d"
Jul 07 23:39:44.421339 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:44.421288    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d"} err="failed to get container status \"2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d\": rpc error: code = NotFound desc = could not find container \"2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d\": container with ID starting with 2f27df0e0876f3198c1a757c5ff1da686bc3c23f9ee048d75696eb500275de1d not found: ID does not exist"
Jul 07 23:39:44.429278 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:44.429253    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"]
Jul 07 23:39:44.435674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:44.435651    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f10e4379-59bd4c58f4-2jgf2"]
Jul 07 23:39:45.963170 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:45.963141    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b186e191-6492-4181-ab75-97633492a077" path="/var/lib/kubelet/pods/b186e191-6492-4181-ab75-97633492a077/volumes"
Jul 07 23:39:49.507791 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.507756    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"]
Jul 07 23:39:49.509037 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.508992    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b186e191-6492-4181-ab75-97633492a077" containerName="maas-api"
Jul 07 23:39:49.509037 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.509026    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b186e191-6492-4181-ab75-97633492a077" containerName="maas-api"
Jul 07 23:39:49.509248 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.509124    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b186e191-6492-4181-ab75-97633492a077" containerName="maas-api"
Jul 07 23:39:49.514368 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.514344    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.517171 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.517101    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9824dcc7-openshift-default-dockercfg-gbzsd\""
Jul 07 23:39:49.525874 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.525047    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"]
Jul 07 23:39:49.553512 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.553474    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-data\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.553758 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.553532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkdv8\" (UniqueName: \"kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-kube-api-access-fkdv8\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.553758 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.553635    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-socket\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.553758 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.553676    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-credential-socket\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.553758 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.553736    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1101a29-fde3-4137-83dc-cf530afc061d-istiod-ca-cert\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.553923 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.553766    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-certs\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.553923 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.553783    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1101a29-fde3-4137-83dc-cf530afc061d-istio-podinfo\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.553923 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.553837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-envoy\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.553923 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.553867    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-istio-token\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.654506 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.654471    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkdv8\" (UniqueName: \"kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-kube-api-access-fkdv8\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.654506 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.654520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-socket\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.654544    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-credential-socket\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.654603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1101a29-fde3-4137-83dc-cf530afc061d-istiod-ca-cert\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.654632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-certs\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.654657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1101a29-fde3-4137-83dc-cf530afc061d-istio-podinfo\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.654702    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-envoy\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.654741    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-istio-token\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.654780    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-data\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.654998    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-socket\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.655131    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-credential-socket\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.655180    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-certs\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.655246    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-data\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.655804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.655503    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1101a29-fde3-4137-83dc-cf530afc061d-istiod-ca-cert\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.657119 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.657094    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-envoy\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.657545 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.657519    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1101a29-fde3-4137-83dc-cf530afc061d-istio-podinfo\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.662314 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.662290    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-istio-token\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.662501 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.662479    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkdv8\" (UniqueName: \"kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-kube-api-access-fkdv8\") pod \"e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") " pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.828990 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.828956    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:49.889642 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.889553    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:49.957358 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.957313    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/493b17a7-e438-41e7-9255-f20718b83336-istio-podinfo\") pod \"493b17a7-e438-41e7-9255-f20718b83336\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") "
Jul 07 23:39:49.957596 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.957375    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/493b17a7-e438-41e7-9255-f20718b83336-istiod-ca-cert\") pod \"493b17a7-e438-41e7-9255-f20718b83336\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") "
Jul 07 23:39:49.957596 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.957424    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-istio-token\") pod \"493b17a7-e438-41e7-9255-f20718b83336\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") "
Jul 07 23:39:49.957596 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.957455    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-data\") pod \"493b17a7-e438-41e7-9255-f20718b83336\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") "
Jul 07 23:39:49.957596 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.957513    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-certs\") pod \"493b17a7-e438-41e7-9255-f20718b83336\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") "
Jul 07 23:39:49.957596 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.957546    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-envoy\") pod \"493b17a7-e438-41e7-9255-f20718b83336\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") "
Jul 07 23:39:49.957855 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.957618    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-credential-socket\") pod \"493b17a7-e438-41e7-9255-f20718b83336\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") "
Jul 07 23:39:49.957855 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.957681    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzxq5\" (UniqueName: \"kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-kube-api-access-pzxq5\") pod \"493b17a7-e438-41e7-9255-f20718b83336\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") "
Jul 07 23:39:49.957855 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.957732    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-socket\") pod \"493b17a7-e438-41e7-9255-f20718b83336\" (UID: \"493b17a7-e438-41e7-9255-f20718b83336\") "
Jul 07 23:39:49.957855 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.957780    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/493b17a7-e438-41e7-9255-f20718b83336-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "493b17a7-e438-41e7-9255-f20718b83336" (UID: "493b17a7-e438-41e7-9255-f20718b83336"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:39:49.958065 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.958029    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/493b17a7-e438-41e7-9255-f20718b83336-istiod-ca-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:49.958484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.958137    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "493b17a7-e438-41e7-9255-f20718b83336" (UID: "493b17a7-e438-41e7-9255-f20718b83336"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:39:49.958484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.958247    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "493b17a7-e438-41e7-9255-f20718b83336" (UID: "493b17a7-e438-41e7-9255-f20718b83336"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:39:49.958484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.958262    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "493b17a7-e438-41e7-9255-f20718b83336" (UID: "493b17a7-e438-41e7-9255-f20718b83336"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:39:49.959306 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.958980    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-data" (OuterVolumeSpecName: "istio-data") pod "493b17a7-e438-41e7-9255-f20718b83336" (UID: "493b17a7-e438-41e7-9255-f20718b83336"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:39:49.960262 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.960212    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-istio-token" (OuterVolumeSpecName: "istio-token") pod "493b17a7-e438-41e7-9255-f20718b83336" (UID: "493b17a7-e438-41e7-9255-f20718b83336"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:39:49.960387 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.960358    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "493b17a7-e438-41e7-9255-f20718b83336" (UID: "493b17a7-e438-41e7-9255-f20718b83336"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:39:49.960503 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.960390    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/493b17a7-e438-41e7-9255-f20718b83336-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "493b17a7-e438-41e7-9255-f20718b83336" (UID: "493b17a7-e438-41e7-9255-f20718b83336"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 23:39:49.960743 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.960703    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-kube-api-access-pzxq5" (OuterVolumeSpecName: "kube-api-access-pzxq5") pod "493b17a7-e438-41e7-9255-f20718b83336" (UID: "493b17a7-e438-41e7-9255-f20718b83336"). InnerVolumeSpecName "kube-api-access-pzxq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:39:49.976860 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.976838    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"]
Jul 07 23:39:49.979203 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:39:49.979172    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1101a29_fde3_4137_83dc_cf530afc061d.slice/crio-ec07b615b0e23e0c2bffc404856064560dcc1719028518f3e4987d8bee14edd3 WatchSource:0}: Error finding container ec07b615b0e23e0c2bffc404856064560dcc1719028518f3e4987d8bee14edd3: Status 404 returned error can't find the container with id ec07b615b0e23e0c2bffc404856064560dcc1719028518f3e4987d8bee14edd3
Jul 07 23:39:49.981280 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.981247    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:39:49.981368 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.981315    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:39:49.981368 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:49.981352    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:39:50.059583 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.059475    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:50.059583 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.059510    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-envoy\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:50.059583 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.059523    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-credential-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:50.059583 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.059538    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzxq5\" (UniqueName: \"kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-kube-api-access-pzxq5\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:50.059583 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.059552    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-workload-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:50.059583 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.059580    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/493b17a7-e438-41e7-9255-f20718b83336-istio-podinfo\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:50.060016 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.059591    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/493b17a7-e438-41e7-9255-f20718b83336-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:50.060016 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.059604    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/493b17a7-e438-41e7-9255-f20718b83336-istio-data\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:50.443243 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.443205    2578 generic.go:356] "Generic (PLEG): container finished" podID="493b17a7-e438-41e7-9255-f20718b83336" containerID="955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5" exitCode=0
Jul 07 23:39:50.443405 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.443252    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"
Jul 07 23:39:50.443405 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.443289    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6" event={"ID":"493b17a7-e438-41e7-9255-f20718b83336","Type":"ContainerDied","Data":"955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5"}
Jul 07 23:39:50.443405 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.443344    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6" event={"ID":"493b17a7-e438-41e7-9255-f20718b83336","Type":"ContainerDied","Data":"a2c375860a773508316b9a2c836a45435ae3efe5780bea527316bb63f2fe03ee"}
Jul 07 23:39:50.443405 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.443370    2578 scope.go:117] "RemoveContainer" containerID="955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5"
Jul 07 23:39:50.445036 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.445007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn" event={"ID":"e1101a29-fde3-4137-83dc-cf530afc061d","Type":"ContainerStarted","Data":"5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b"}
Jul 07 23:39:50.445143 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.445058    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn" event={"ID":"e1101a29-fde3-4137-83dc-cf530afc061d","Type":"ContainerStarted","Data":"ec07b615b0e23e0c2bffc404856064560dcc1719028518f3e4987d8bee14edd3"}
Jul 07 23:39:50.452370 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.452347    2578 scope.go:117] "RemoveContainer" containerID="955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5"
Jul 07 23:39:50.452690 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:39:50.452667    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5\": container with ID starting with 955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5 not found: ID does not exist" containerID="955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5"
Jul 07 23:39:50.452772 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.452702    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5"} err="failed to get container status \"955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5\": rpc error: code = NotFound desc = could not find container \"955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5\": container with ID starting with 955e2e3c1ddd41058e512564106efadd7fe239c5f4cd0b4406e34d389ab6f0f5 not found: ID does not exist"
Jul 07 23:39:50.464446 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.464415    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"]
Jul 07 23:39:50.468851 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.468827    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f10e4379-openshift-default-8666bccf49-ph4b6"]
Jul 07 23:39:50.486491 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.486437    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn" podStartSLOduration=1.486417395 podStartE2EDuration="1.486417395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:39:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:39:50.484212257 +0000 UTC m=+1575.069781618" watchObservedRunningTime="2026-07-07 23:39:50.486417395 +0000 UTC m=+1575.071986755"
Jul 07 23:39:50.786423 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.786328    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"]
Jul 07 23:39:50.787555 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.787522    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="493b17a7-e438-41e7-9255-f20718b83336" containerName="istio-proxy"
Jul 07 23:39:50.787555 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.787555    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="493b17a7-e438-41e7-9255-f20718b83336" containerName="istio-proxy"
Jul 07 23:39:50.787774 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.787686    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="493b17a7-e438-41e7-9255-f20718b83336" containerName="istio-proxy"
Jul 07 23:39:50.790667 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.790647    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:50.793285 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.793247    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9824dcc7-serving-cert\""
Jul 07 23:39:50.802281 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.802192    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"]
Jul 07 23:39:50.829252 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.829217    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:50.867181 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.867149    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0404e386-519f-4db2-b3b0-1c1a79805e3b-openshift-service-ca\") pod \"maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:50.867337 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.867201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0404e386-519f-4db2-b3b0-1c1a79805e3b-maas-api-tls\") pod \"maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:50.867398 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.867333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhwff\" (UniqueName: \"kubernetes.io/projected/0404e386-519f-4db2-b3b0-1c1a79805e3b-kube-api-access-hhwff\") pod \"maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:50.968855 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.968818    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhwff\" (UniqueName: \"kubernetes.io/projected/0404e386-519f-4db2-b3b0-1c1a79805e3b-kube-api-access-hhwff\") pod \"maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:50.969076 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.968888    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0404e386-519f-4db2-b3b0-1c1a79805e3b-openshift-service-ca\") pod \"maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:50.969076 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.968931    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0404e386-519f-4db2-b3b0-1c1a79805e3b-maas-api-tls\") pod \"maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:50.970356 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.970247    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0404e386-519f-4db2-b3b0-1c1a79805e3b-openshift-service-ca\") pod \"maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:50.971954 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.971927    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0404e386-519f-4db2-b3b0-1c1a79805e3b-maas-api-tls\") pod \"maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:50.977783 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:50.977754    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhwff\" (UniqueName: \"kubernetes.io/projected/0404e386-519f-4db2-b3b0-1c1a79805e3b-kube-api-access-hhwff\") pod \"maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:51.107202 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:51.107106    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:51.259445 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:51.259418    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"]
Jul 07 23:39:51.261275 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:39:51.261245    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0404e386_519f_4db2_b3b0_1c1a79805e3b.slice/crio-78629c1ba47c919276ae66941c592dda9b3ed81376bed1e85de42ae7dd0e669a WatchSource:0}: Error finding container 78629c1ba47c919276ae66941c592dda9b3ed81376bed1e85de42ae7dd0e669a: Status 404 returned error can't find the container with id 78629c1ba47c919276ae66941c592dda9b3ed81376bed1e85de42ae7dd0e669a
Jul 07 23:39:51.459137 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:51.459096    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp" event={"ID":"0404e386-519f-4db2-b3b0-1c1a79805e3b","Type":"ContainerStarted","Data":"78629c1ba47c919276ae66941c592dda9b3ed81376bed1e85de42ae7dd0e669a"}
Jul 07 23:39:51.833948 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:51.833865    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:51.963531 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:51.963491    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="493b17a7-e438-41e7-9255-f20718b83336" path="/var/lib/kubelet/pods/493b17a7-e438-41e7-9255-f20718b83336/volumes"
Jul 07 23:39:52.465628 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:52.465586    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp" event={"ID":"0404e386-519f-4db2-b3b0-1c1a79805e3b","Type":"ContainerStarted","Data":"403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056"}
Jul 07 23:39:52.465863 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:52.465843    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:52.465923 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:52.465891    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:52.466751 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:52.466730    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:39:52.483532 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:52.483480    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp" podStartSLOduration=2.10140648 podStartE2EDuration="2.483463211s" totalImagesPullingTime="382.056731ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:39:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:39:52.481899207 +0000 UTC m=+1577.067468570" watchObservedRunningTime="2026-07-07 23:39:52.483463211 +0000 UTC m=+1577.069032573"
Jul 07 23:39:56.022154 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.022114    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"]
Jul 07 23:39:56.022657 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.022380    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp" podUID="0404e386-519f-4db2-b3b0-1c1a79805e3b" containerName="maas-api" containerID="cri-o://403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056" gracePeriod=30
Jul 07 23:39:56.029036 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.029012    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:56.285598 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.285510    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:56.322978 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.322940    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0404e386-519f-4db2-b3b0-1c1a79805e3b-openshift-service-ca\") pod \"0404e386-519f-4db2-b3b0-1c1a79805e3b\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") "
Jul 07 23:39:56.323153 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.323059    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0404e386-519f-4db2-b3b0-1c1a79805e3b-maas-api-tls\") pod \"0404e386-519f-4db2-b3b0-1c1a79805e3b\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") "
Jul 07 23:39:56.323153 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.323112    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhwff\" (UniqueName: \"kubernetes.io/projected/0404e386-519f-4db2-b3b0-1c1a79805e3b-kube-api-access-hhwff\") pod \"0404e386-519f-4db2-b3b0-1c1a79805e3b\" (UID: \"0404e386-519f-4db2-b3b0-1c1a79805e3b\") "
Jul 07 23:39:56.323413 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.323365    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0404e386-519f-4db2-b3b0-1c1a79805e3b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0404e386-519f-4db2-b3b0-1c1a79805e3b" (UID: "0404e386-519f-4db2-b3b0-1c1a79805e3b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:39:56.325278 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.325250    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0404e386-519f-4db2-b3b0-1c1a79805e3b-kube-api-access-hhwff" (OuterVolumeSpecName: "kube-api-access-hhwff") pod "0404e386-519f-4db2-b3b0-1c1a79805e3b" (UID: "0404e386-519f-4db2-b3b0-1c1a79805e3b"). InnerVolumeSpecName "kube-api-access-hhwff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:39:56.325376 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.325252    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0404e386-519f-4db2-b3b0-1c1a79805e3b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0404e386-519f-4db2-b3b0-1c1a79805e3b" (UID: "0404e386-519f-4db2-b3b0-1c1a79805e3b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:39:56.424173 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.424138    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0404e386-519f-4db2-b3b0-1c1a79805e3b-openshift-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:56.424173 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.424172    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0404e386-519f-4db2-b3b0-1c1a79805e3b-maas-api-tls\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:56.424173 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.424183    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhwff\" (UniqueName: \"kubernetes.io/projected/0404e386-519f-4db2-b3b0-1c1a79805e3b-kube-api-access-hhwff\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:39:56.486973 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.486441    2578 generic.go:356] "Generic (PLEG): container finished" podID="0404e386-519f-4db2-b3b0-1c1a79805e3b" containerID="403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056" exitCode=0
Jul 07 23:39:56.486973 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.486499    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"
Jul 07 23:39:56.486973 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.486633    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp" event={"ID":"0404e386-519f-4db2-b3b0-1c1a79805e3b","Type":"ContainerDied","Data":"403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056"}
Jul 07 23:39:56.486973 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.486739    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp" event={"ID":"0404e386-519f-4db2-b3b0-1c1a79805e3b","Type":"ContainerDied","Data":"78629c1ba47c919276ae66941c592dda9b3ed81376bed1e85de42ae7dd0e669a"}
Jul 07 23:39:56.486973 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.486784    2578 scope.go:117] "RemoveContainer" containerID="403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056"
Jul 07 23:39:56.496377 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.496359    2578 scope.go:117] "RemoveContainer" containerID="403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056"
Jul 07 23:39:56.496703 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:39:56.496683    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056\": container with ID starting with 403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056 not found: ID does not exist" containerID="403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056"
Jul 07 23:39:56.496777 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.496710    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056"} err="failed to get container status \"403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056\": rpc error: code = NotFound desc = could not find container \"403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056\": container with ID starting with 403ca8fb8ff4479b37b88e9f99e0f1309bb5b181086fd211d319cf15a9195056 not found: ID does not exist"
Jul 07 23:39:56.509116 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.509090    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"]
Jul 07 23:39:56.512250 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:56.512225    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9824dcc7-fcccd9d79-5bjkp"]
Jul 07 23:39:57.962683 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:39:57.962640    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0404e386-519f-4db2-b3b0-1c1a79805e3b" path="/var/lib/kubelet/pods/0404e386-519f-4db2-b3b0-1c1a79805e3b/volumes"
Jul 07 23:40:01.793309 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:01.793275    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"]
Jul 07 23:40:01.793709 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:01.793475    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn" podUID="e1101a29-fde3-4137-83dc-cf530afc061d" containerName="istio-proxy" containerID="cri-o://5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b" gracePeriod=30
Jul 07 23:40:07.054258 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.054197    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:40:07.119109 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119073    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-socket\") pod \"e1101a29-fde3-4137-83dc-cf530afc061d\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") "
Jul 07 23:40:07.119109 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119113    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1101a29-fde3-4137-83dc-cf530afc061d-istio-podinfo\") pod \"e1101a29-fde3-4137-83dc-cf530afc061d\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") "
Jul 07 23:40:07.119354 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119155    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-data\") pod \"e1101a29-fde3-4137-83dc-cf530afc061d\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") "
Jul 07 23:40:07.119354 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119177    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-certs\") pod \"e1101a29-fde3-4137-83dc-cf530afc061d\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") "
Jul 07 23:40:07.119354 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119193    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-envoy\") pod \"e1101a29-fde3-4137-83dc-cf530afc061d\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") "
Jul 07 23:40:07.119354 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119230    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1101a29-fde3-4137-83dc-cf530afc061d-istiod-ca-cert\") pod \"e1101a29-fde3-4137-83dc-cf530afc061d\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") "
Jul 07 23:40:07.119993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119399    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkdv8\" (UniqueName: \"kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-kube-api-access-fkdv8\") pod \"e1101a29-fde3-4137-83dc-cf530afc061d\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") "
Jul 07 23:40:07.119993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119401    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e1101a29-fde3-4137-83dc-cf530afc061d" (UID: "e1101a29-fde3-4137-83dc-cf530afc061d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:40:07.119993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119453    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-istio-token\") pod \"e1101a29-fde3-4137-83dc-cf530afc061d\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") "
Jul 07 23:40:07.119993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119476    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-data" (OuterVolumeSpecName: "istio-data") pod "e1101a29-fde3-4137-83dc-cf530afc061d" (UID: "e1101a29-fde3-4137-83dc-cf530afc061d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:40:07.119993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119516    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-credential-socket\") pod \"e1101a29-fde3-4137-83dc-cf530afc061d\" (UID: \"e1101a29-fde3-4137-83dc-cf530afc061d\") "
Jul 07 23:40:07.119993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119720    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e1101a29-fde3-4137-83dc-cf530afc061d" (UID: "e1101a29-fde3-4137-83dc-cf530afc061d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:40:07.119993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119859    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e1101a29-fde3-4137-83dc-cf530afc061d" (UID: "e1101a29-fde3-4137-83dc-cf530afc061d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:40:07.119993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119943    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-data\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:07.119993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119965    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:07.119993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119980    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-credential-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:07.119993 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.119995    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-workload-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:07.120382 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.120064    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1101a29-fde3-4137-83dc-cf530afc061d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e1101a29-fde3-4137-83dc-cf530afc061d" (UID: "e1101a29-fde3-4137-83dc-cf530afc061d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:40:07.121977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.121937    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-kube-api-access-fkdv8" (OuterVolumeSpecName: "kube-api-access-fkdv8") pod "e1101a29-fde3-4137-83dc-cf530afc061d" (UID: "e1101a29-fde3-4137-83dc-cf530afc061d"). InnerVolumeSpecName "kube-api-access-fkdv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:40:07.121977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.121954    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e1101a29-fde3-4137-83dc-cf530afc061d" (UID: "e1101a29-fde3-4137-83dc-cf530afc061d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:40:07.122168 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.122044    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e1101a29-fde3-4137-83dc-cf530afc061d" (UID: "e1101a29-fde3-4137-83dc-cf530afc061d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:40:07.122223 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.122187    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1101a29-fde3-4137-83dc-cf530afc061d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e1101a29-fde3-4137-83dc-cf530afc061d" (UID: "e1101a29-fde3-4137-83dc-cf530afc061d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 23:40:07.220746 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.220713    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1101a29-fde3-4137-83dc-cf530afc061d-istio-podinfo\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:07.220746 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.220742    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1101a29-fde3-4137-83dc-cf530afc061d-istio-envoy\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:07.220746 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.220750    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1101a29-fde3-4137-83dc-cf530afc061d-istiod-ca-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:07.220986 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.220759    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkdv8\" (UniqueName: \"kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-kube-api-access-fkdv8\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:07.220986 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.220768    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1101a29-fde3-4137-83dc-cf530afc061d-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:07.544541 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.544506    2578 generic.go:356] "Generic (PLEG): container finished" podID="e1101a29-fde3-4137-83dc-cf530afc061d" containerID="5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b" exitCode=0
Jul 07 23:40:07.544752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.544551    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"
Jul 07 23:40:07.544752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.544596    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn" event={"ID":"e1101a29-fde3-4137-83dc-cf530afc061d","Type":"ContainerDied","Data":"5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b"}
Jul 07 23:40:07.544752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.544653    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn" event={"ID":"e1101a29-fde3-4137-83dc-cf530afc061d","Type":"ContainerDied","Data":"ec07b615b0e23e0c2bffc404856064560dcc1719028518f3e4987d8bee14edd3"}
Jul 07 23:40:07.544752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.544680    2578 scope.go:117] "RemoveContainer" containerID="5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b"
Jul 07 23:40:07.553739 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.553713    2578 scope.go:117] "RemoveContainer" containerID="5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b"
Jul 07 23:40:07.554028 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:40:07.554009    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b\": container with ID starting with 5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b not found: ID does not exist" containerID="5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b"
Jul 07 23:40:07.554105 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.554039    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b"} err="failed to get container status \"5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b\": rpc error: code = NotFound desc = could not find container \"5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b\": container with ID starting with 5d3e4080a9e0a9756c3ae38d167fee00d94d7ce211b24ecbfa16b9f9f2ba815b not found: ID does not exist"
Jul 07 23:40:07.565671 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.565644    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"]
Jul 07 23:40:07.571824 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.571796    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9824dcc7-openshift-default-5d786c9787-6c8zn"]
Jul 07 23:40:07.963731 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:07.963698    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1101a29-fde3-4137-83dc-cf530afc061d" path="/var/lib/kubelet/pods/e1101a29-fde3-4137-83dc-cf530afc061d/volumes"
Jul 07 23:40:08.170297 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.170260    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"]
Jul 07 23:40:08.170916 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.170862    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0404e386-519f-4db2-b3b0-1c1a79805e3b" containerName="maas-api"
Jul 07 23:40:08.171080 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.170920    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0404e386-519f-4db2-b3b0-1c1a79805e3b" containerName="maas-api"
Jul 07 23:40:08.171080 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.170946    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1101a29-fde3-4137-83dc-cf530afc061d" containerName="istio-proxy"
Jul 07 23:40:08.171080 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.170955    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1101a29-fde3-4137-83dc-cf530afc061d" containerName="istio-proxy"
Jul 07 23:40:08.171282 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.171085    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0404e386-519f-4db2-b3b0-1c1a79805e3b" containerName="maas-api"
Jul 07 23:40:08.171282 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.171101    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1101a29-fde3-4137-83dc-cf530afc061d" containerName="istio-proxy"
Jul 07 23:40:08.176025 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.176002    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.178598 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.178553    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8e718b57-openshift-default-dockercfg-44mhb\""
Jul 07 23:40:08.186249 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.186220    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"]
Jul 07 23:40:08.230057 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.229956    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-token\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.230057 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.230021    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-data\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.230266 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.230081    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-credential-socket\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.230266 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.230107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-certs\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.230266 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.230167    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istiod-ca-cert\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.230266 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.230214    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-podinfo\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.230266 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.230247    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-socket\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.230436 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.230295    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zvw5\" (UniqueName: \"kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-kube-api-access-6zvw5\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.230436 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.230328    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-envoy\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.331699 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.331650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-envoy\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.331699 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.331704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-token\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.331964 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.331737    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-data\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.331964 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.331759    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-credential-socket\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.331964 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.331773    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-certs\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.331964 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.331802    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istiod-ca-cert\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.331964 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.331829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-podinfo\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.331964 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.331862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-socket\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.331964 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.331895    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zvw5\" (UniqueName: \"kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-kube-api-access-6zvw5\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.332326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.332230    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-data\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.332326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.332249    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-certs\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.332326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.332319    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-credential-socket\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.332499 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.332474    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-socket\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.332661 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.332646    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istiod-ca-cert\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.334232 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.334208    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-envoy\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.334332 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.334293    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-podinfo\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.340462 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.340437    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-token\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.340612 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.340594    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zvw5\" (UniqueName: \"kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-kube-api-access-6zvw5\") pod \"e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") " pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.489681 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.489599    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:08.628719 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.628695    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"]
Jul 07 23:40:08.630629 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:40:08.630603    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab37b01_637a_4fc6_b926_dcb42f771dbb.slice/crio-192029dd2eeacf82705e39cceacdce4c949fe4acba46a76de692c814d5a5c48e WatchSource:0}: Error finding container 192029dd2eeacf82705e39cceacdce4c949fe4acba46a76de692c814d5a5c48e: Status 404 returned error can't find the container with id 192029dd2eeacf82705e39cceacdce4c949fe4acba46a76de692c814d5a5c48e
Jul 07 23:40:08.632738 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.632705    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:40:08.632823 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.632771    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:40:08.632823 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:08.632801    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:40:09.375938 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.375900    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"]
Jul 07 23:40:09.380114 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.379957    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:09.382614 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.382588    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-8e718b57-serving-cert\""
Jul 07 23:40:09.389488 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.389464    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"]
Jul 07 23:40:09.440202 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.440157    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s442h\" (UniqueName: \"kubernetes.io/projected/818f6db1-4ade-456e-b7b8-90bd239fa834-kube-api-access-s442h\") pod \"maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:09.440386 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.440216    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/818f6db1-4ade-456e-b7b8-90bd239fa834-openshift-service-ca\") pod \"maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:09.440386 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.440294    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818f6db1-4ade-456e-b7b8-90bd239fa834-maas-api-tls\") pod \"maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:09.541879 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.541841    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s442h\" (UniqueName: \"kubernetes.io/projected/818f6db1-4ade-456e-b7b8-90bd239fa834-kube-api-access-s442h\") pod \"maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:09.542058 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.541884    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/818f6db1-4ade-456e-b7b8-90bd239fa834-openshift-service-ca\") pod \"maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:09.542058 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.541913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818f6db1-4ade-456e-b7b8-90bd239fa834-maas-api-tls\") pod \"maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:09.542200 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:40:09.542073    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-serving-cert: secret "maas-api-e2e-derive-8e718b57-serving-cert" not found
Jul 07 23:40:09.542200 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:40:09.542137    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818f6db1-4ade-456e-b7b8-90bd239fa834-maas-api-tls podName:818f6db1-4ade-456e-b7b8-90bd239fa834 nodeName:}" failed. No retries permitted until 2026-07-07 23:40:10.042120136 +0000 UTC m=+1594.627689475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/818f6db1-4ade-456e-b7b8-90bd239fa834-maas-api-tls") pod "maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz" (UID: "818f6db1-4ade-456e-b7b8-90bd239fa834") : secret "maas-api-e2e-derive-8e718b57-serving-cert" not found
Jul 07 23:40:09.542492 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.542473    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/818f6db1-4ade-456e-b7b8-90bd239fa834-openshift-service-ca\") pod \"maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:09.553819 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.553783    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s442h\" (UniqueName: \"kubernetes.io/projected/818f6db1-4ade-456e-b7b8-90bd239fa834-kube-api-access-s442h\") pod \"maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:09.559808 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.559780    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7" event={"ID":"3ab37b01-637a-4fc6-b926-dcb42f771dbb","Type":"ContainerStarted","Data":"3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6"}
Jul 07 23:40:09.559962 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.559827    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7" event={"ID":"3ab37b01-637a-4fc6-b926-dcb42f771dbb","Type":"ContainerStarted","Data":"192029dd2eeacf82705e39cceacdce4c949fe4acba46a76de692c814d5a5c48e"}
Jul 07 23:40:09.585010 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:09.584955    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7" podStartSLOduration=1.5849382109999999 podStartE2EDuration="1.584938211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:40:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:40:09.583201204 +0000 UTC m=+1594.168770566" watchObservedRunningTime="2026-07-07 23:40:09.584938211 +0000 UTC m=+1594.170507571"
Jul 07 23:40:10.046748 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:10.046709    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818f6db1-4ade-456e-b7b8-90bd239fa834-maas-api-tls\") pod \"maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:10.049271 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:10.049239    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818f6db1-4ade-456e-b7b8-90bd239fa834-maas-api-tls\") pod \"maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:10.293279 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:10.293235    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:10.435081 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:10.435054    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"]
Jul 07 23:40:10.437271 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:40:10.437242    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818f6db1_4ade_456e_b7b8_90bd239fa834.slice/crio-0169914d38b6a4dcb833626cf725960cdf090420fc4fe7caa113cb5ed4b96280 WatchSource:0}: Error finding container 0169914d38b6a4dcb833626cf725960cdf090420fc4fe7caa113cb5ed4b96280: Status 404 returned error can't find the container with id 0169914d38b6a4dcb833626cf725960cdf090420fc4fe7caa113cb5ed4b96280
Jul 07 23:40:10.490535 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:10.490506    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:10.495403 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:10.495377    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:10.566033 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:10.565947    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz" event={"ID":"818f6db1-4ade-456e-b7b8-90bd239fa834","Type":"ContainerStarted","Data":"0169914d38b6a4dcb833626cf725960cdf090420fc4fe7caa113cb5ed4b96280"}
Jul 07 23:40:10.566227 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:10.566211    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:10.567306 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:10.567286    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:11.572840 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:11.572805    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz" event={"ID":"818f6db1-4ade-456e-b7b8-90bd239fa834","Type":"ContainerStarted","Data":"fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322"}
Jul 07 23:40:11.573290 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:11.572869    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:11.591126 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:11.591075    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz" podStartSLOduration=2.189507778 podStartE2EDuration="2.591057465s" totalImagesPullingTime="401.549687ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:40:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:40:11.590116822 +0000 UTC m=+1596.175686194" watchObservedRunningTime="2026-07-07 23:40:11.591057465 +0000 UTC m=+1596.176626827"
Jul 07 23:40:13.923033 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:13.922985    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"]
Jul 07 23:40:13.923425 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:13.923203    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7" podUID="3ab37b01-637a-4fc6-b926-dcb42f771dbb" containerName="istio-proxy" containerID="cri-o://3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6" gracePeriod=30
Jul 07 23:40:14.107420 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.107376    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"]
Jul 07 23:40:14.107647 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.107613    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz" podUID="818f6db1-4ade-456e-b7b8-90bd239fa834" containerName="maas-api" containerID="cri-o://fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322" gracePeriod=30
Jul 07 23:40:14.376023 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.375992    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:14.503183 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.501848    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s442h\" (UniqueName: \"kubernetes.io/projected/818f6db1-4ade-456e-b7b8-90bd239fa834-kube-api-access-s442h\") pod \"818f6db1-4ade-456e-b7b8-90bd239fa834\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") "
Jul 07 23:40:14.503183 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.501978    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818f6db1-4ade-456e-b7b8-90bd239fa834-maas-api-tls\") pod \"818f6db1-4ade-456e-b7b8-90bd239fa834\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") "
Jul 07 23:40:14.503183 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.502130    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/818f6db1-4ade-456e-b7b8-90bd239fa834-openshift-service-ca\") pod \"818f6db1-4ade-456e-b7b8-90bd239fa834\" (UID: \"818f6db1-4ade-456e-b7b8-90bd239fa834\") "
Jul 07 23:40:14.503183 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.502807    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/818f6db1-4ade-456e-b7b8-90bd239fa834-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "818f6db1-4ade-456e-b7b8-90bd239fa834" (UID: "818f6db1-4ade-456e-b7b8-90bd239fa834"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:40:14.507722 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.507664    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/818f6db1-4ade-456e-b7b8-90bd239fa834-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "818f6db1-4ade-456e-b7b8-90bd239fa834" (UID: "818f6db1-4ade-456e-b7b8-90bd239fa834"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:40:14.508361 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.508331    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/818f6db1-4ade-456e-b7b8-90bd239fa834-kube-api-access-s442h" (OuterVolumeSpecName: "kube-api-access-s442h") pod "818f6db1-4ade-456e-b7b8-90bd239fa834" (UID: "818f6db1-4ade-456e-b7b8-90bd239fa834"). InnerVolumeSpecName "kube-api-access-s442h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:40:14.595859 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.595816    2578 generic.go:356] "Generic (PLEG): container finished" podID="818f6db1-4ade-456e-b7b8-90bd239fa834" containerID="fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322" exitCode=0
Jul 07 23:40:14.596068 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.595987    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz" event={"ID":"818f6db1-4ade-456e-b7b8-90bd239fa834","Type":"ContainerDied","Data":"fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322"}
Jul 07 23:40:14.596068 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.596043    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz" event={"ID":"818f6db1-4ade-456e-b7b8-90bd239fa834","Type":"ContainerDied","Data":"0169914d38b6a4dcb833626cf725960cdf090420fc4fe7caa113cb5ed4b96280"}
Jul 07 23:40:14.596192 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.596075    2578 scope.go:117] "RemoveContainer" containerID="fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322"
Jul 07 23:40:14.596252 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.596242    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"
Jul 07 23:40:14.604287 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.604255    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s442h\" (UniqueName: \"kubernetes.io/projected/818f6db1-4ade-456e-b7b8-90bd239fa834-kube-api-access-s442h\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:14.604544 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.604499    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818f6db1-4ade-456e-b7b8-90bd239fa834-maas-api-tls\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:14.604695 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.604523    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/818f6db1-4ade-456e-b7b8-90bd239fa834-openshift-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:14.612707 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.612684    2578 scope.go:117] "RemoveContainer" containerID="fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322"
Jul 07 23:40:14.612997 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:40:14.612977    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322\": container with ID starting with fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322 not found: ID does not exist" containerID="fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322"
Jul 07 23:40:14.613067 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.613011    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322"} err="failed to get container status \"fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322\": rpc error: code = NotFound desc = could not find container \"fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322\": container with ID starting with fce9e4a0975390dd16fecccd7b17e0e2225f7de622e6af948a14b07c140cb322 not found: ID does not exist"
Jul 07 23:40:14.623943 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.623912    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"]
Jul 07 23:40:14.627745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:14.627717    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e718b57-6bd5c4bfc-c8cfz"]
Jul 07 23:40:15.962383 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:15.962353    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="818f6db1-4ade-456e-b7b8-90bd239fa834" path="/var/lib/kubelet/pods/818f6db1-4ade-456e-b7b8-90bd239fa834/volumes"
Jul 07 23:40:19.980866 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:19.980840    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:20.163842 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.163789    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-token\") pod \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") "
Jul 07 23:40:20.164055 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.163867    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zvw5\" (UniqueName: \"kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-kube-api-access-6zvw5\") pod \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") "
Jul 07 23:40:20.164055 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.163905    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-certs\") pod \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") "
Jul 07 23:40:20.164055 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.163936    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-socket\") pod \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") "
Jul 07 23:40:20.164055 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164027    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-credential-socket\") pod \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") "
Jul 07 23:40:20.164288 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164119    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-data\") pod \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") "
Jul 07 23:40:20.164288 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164151    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-envoy\") pod \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") "
Jul 07 23:40:20.164288 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164190    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-podinfo\") pod \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") "
Jul 07 23:40:20.164288 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164218    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ab37b01-637a-4fc6-b926-dcb42f771dbb" (UID: "3ab37b01-637a-4fc6-b926-dcb42f771dbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:40:20.164288 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164226    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ab37b01-637a-4fc6-b926-dcb42f771dbb" (UID: "3ab37b01-637a-4fc6-b926-dcb42f771dbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:40:20.164288 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164249    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istiod-ca-cert\") pod \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\" (UID: \"3ab37b01-637a-4fc6-b926-dcb42f771dbb\") "
Jul 07 23:40:20.164598 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164527    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ab37b01-637a-4fc6-b926-dcb42f771dbb" (UID: "3ab37b01-637a-4fc6-b926-dcb42f771dbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:40:20.164598 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164535    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:20.164598 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164553    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-workload-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:20.164781 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164604    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ab37b01-637a-4fc6-b926-dcb42f771dbb" (UID: "3ab37b01-637a-4fc6-b926-dcb42f771dbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:40:20.164942 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.164921    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ab37b01-637a-4fc6-b926-dcb42f771dbb" (UID: "3ab37b01-637a-4fc6-b926-dcb42f771dbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:40:20.166228 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.166197    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ab37b01-637a-4fc6-b926-dcb42f771dbb" (UID: "3ab37b01-637a-4fc6-b926-dcb42f771dbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 23:40:20.166377 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.166359    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ab37b01-637a-4fc6-b926-dcb42f771dbb" (UID: "3ab37b01-637a-4fc6-b926-dcb42f771dbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:40:20.166543 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.166521    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-kube-api-access-6zvw5" (OuterVolumeSpecName: "kube-api-access-6zvw5") pod "3ab37b01-637a-4fc6-b926-dcb42f771dbb" (UID: "3ab37b01-637a-4fc6-b926-dcb42f771dbb"). InnerVolumeSpecName "kube-api-access-6zvw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:40:20.166730 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.166548    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ab37b01-637a-4fc6-b926-dcb42f771dbb" (UID: "3ab37b01-637a-4fc6-b926-dcb42f771dbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:40:20.265172 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.265134    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-credential-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:20.265172 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.265165    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-data\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:20.265172 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.265176    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-envoy\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:20.265172 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.265184    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-podinfo\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:20.265452 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.265192    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istiod-ca-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:20.265452 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.265200    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:20.265452 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.265208    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zvw5\" (UniqueName: \"kubernetes.io/projected/3ab37b01-637a-4fc6-b926-dcb42f771dbb-kube-api-access-6zvw5\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:40:20.635896 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.635863    2578 generic.go:356] "Generic (PLEG): container finished" podID="3ab37b01-637a-4fc6-b926-dcb42f771dbb" containerID="3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6" exitCode=0
Jul 07 23:40:20.636078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.635915    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"
Jul 07 23:40:20.636078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.635931    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7" event={"ID":"3ab37b01-637a-4fc6-b926-dcb42f771dbb","Type":"ContainerDied","Data":"3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6"}
Jul 07 23:40:20.636078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.635975    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7" event={"ID":"3ab37b01-637a-4fc6-b926-dcb42f771dbb","Type":"ContainerDied","Data":"192029dd2eeacf82705e39cceacdce4c949fe4acba46a76de692c814d5a5c48e"}
Jul 07 23:40:20.636078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.635997    2578 scope.go:117] "RemoveContainer" containerID="3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6"
Jul 07 23:40:20.645443 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.645419    2578 scope.go:117] "RemoveContainer" containerID="3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6"
Jul 07 23:40:20.645724 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:40:20.645705    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6\": container with ID starting with 3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6 not found: ID does not exist" containerID="3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6"
Jul 07 23:40:20.645796 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.645732    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6"} err="failed to get container status \"3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6\": rpc error: code = NotFound desc = could not find container \"3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6\": container with ID starting with 3578d44d141fb88ff57cfbcf1ce7c0d8eacb07712dd27af029d6421a690736a6 not found: ID does not exist"
Jul 07 23:40:20.657855 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.657824    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"]
Jul 07 23:40:20.661885 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:20.661858    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8e718b57-openshift-default-847dbb7d47-ml6c7"]
Jul 07 23:40:21.963309 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:40:21.963279    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ab37b01-637a-4fc6-b926-dcb42f771dbb" path="/var/lib/kubelet/pods/3ab37b01-637a-4fc6-b926-dcb42f771dbb/volumes"
Jul 07 23:41:56.175974 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.175938    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"]
Jul 07 23:41:56.176513 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.176498    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="818f6db1-4ade-456e-b7b8-90bd239fa834" containerName="maas-api"
Jul 07 23:41:56.176580 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.176516    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="818f6db1-4ade-456e-b7b8-90bd239fa834" containerName="maas-api"
Jul 07 23:41:56.176580 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.176548    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ab37b01-637a-4fc6-b926-dcb42f771dbb" containerName="istio-proxy"
Jul 07 23:41:56.176580 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.176571    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ab37b01-637a-4fc6-b926-dcb42f771dbb" containerName="istio-proxy"
Jul 07 23:41:56.176689 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.176671    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ab37b01-637a-4fc6-b926-dcb42f771dbb" containerName="istio-proxy"
Jul 07 23:41:56.176689 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.176686    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="818f6db1-4ade-456e-b7b8-90bd239fa834" containerName="maas-api"
Jul 07 23:41:56.180190 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.180171    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.183131 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.183105    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-27a7c412-openshift-default-dockercfg-w22nf\""
Jul 07 23:41:56.210370 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.210330    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"]
Jul 07 23:41:56.337049 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.337011    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-socket\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.337390 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.337070    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/440142f5-70f0-4c77-8d99-06288f43255e-istiod-ca-cert\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.337390 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.337088    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-istio-token\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.337390 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.337109    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-credential-socket\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.337390 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.337166    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mtmv\" (UniqueName: \"kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-kube-api-access-5mtmv\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.337390 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.337249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-data\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.337390 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.337288    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-certs\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.337390 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.337337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-envoy\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.337723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.337392    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/440142f5-70f0-4c77-8d99-06288f43255e-istio-podinfo\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.438752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.438649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/440142f5-70f0-4c77-8d99-06288f43255e-istiod-ca-cert\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.438752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.438695    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-istio-token\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.438752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.438717    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-credential-socket\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.439040 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.438817    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mtmv\" (UniqueName: \"kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-kube-api-access-5mtmv\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.439040 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.438842    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-data\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.439040 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.438859    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-certs\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.439040 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.438898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-envoy\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.439040 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.438934    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/440142f5-70f0-4c77-8d99-06288f43255e-istio-podinfo\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.439040 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.439008    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-socket\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.439337 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.439255    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-data\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.439395 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.439338    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-certs\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.439447 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.439392    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-credential-socket\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.439447 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.439439    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-socket\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.439733 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.439709    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/440142f5-70f0-4c77-8d99-06288f43255e-istiod-ca-cert\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.441284 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.441263    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-envoy\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.441431 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.441412    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/440142f5-70f0-4c77-8d99-06288f43255e-istio-podinfo\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.447229 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.447204    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-istio-token\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.447371 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.447340    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mtmv\" (UniqueName: \"kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-kube-api-access-5mtmv\") pod \"e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") " pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.491265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.491229    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:56.640239 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.640203    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"]
Jul 07 23:41:56.642077 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:41:56.642047    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440142f5_70f0_4c77_8d99_06288f43255e.slice/crio-06fa5fe237cc74f218d882797c026b9990901707a265217aa71076b69a277dd1 WatchSource:0}: Error finding container 06fa5fe237cc74f218d882797c026b9990901707a265217aa71076b69a277dd1: Status 404 returned error can't find the container with id 06fa5fe237cc74f218d882797c026b9990901707a265217aa71076b69a277dd1
Jul 07 23:41:56.644377 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.644342    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:41:56.644477 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.644428    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:41:56.644542 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:56.644472    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:41:57.102367 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:57.102256    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" event={"ID":"440142f5-70f0-4c77-8d99-06288f43255e","Type":"ContainerStarted","Data":"acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d"}
Jul 07 23:41:57.102367 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:57.102309    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" event={"ID":"440142f5-70f0-4c77-8d99-06288f43255e","Type":"ContainerStarted","Data":"06fa5fe237cc74f218d882797c026b9990901707a265217aa71076b69a277dd1"}
Jul 07 23:41:57.138465 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:57.138415    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" podStartSLOduration=1.13839706 podStartE2EDuration="1.13839706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:41:57.137126342 +0000 UTC m=+1701.722695702" watchObservedRunningTime="2026-07-07 23:41:57.13839706 +0000 UTC m=+1701.723966421"
Jul 07 23:41:57.492225 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:57.492187    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:57.497498 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:57.497470    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:58.107229 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:58.107198    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:58.108117 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:58.108094    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:41:59.360733 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.360699    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"]
Jul 07 23:41:59.364266 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.364248    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:41:59.365550 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.365519    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d68d71df-1fe2-4592-aba7-28d00d0472be-maas-api-tls\") pod \"maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:41:59.365685 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.365631    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68d71df-1fe2-4592-aba7-28d00d0472be-openshift-service-ca\") pod \"maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:41:59.365685 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.365671    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qh4x\" (UniqueName: \"kubernetes.io/projected/d68d71df-1fe2-4592-aba7-28d00d0472be-kube-api-access-8qh4x\") pod \"maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:41:59.368732 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.368700    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-27a7c412-serving-cert\""
Jul 07 23:41:59.385442 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.385408    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"]
Jul 07 23:41:59.466292 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.466260    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d68d71df-1fe2-4592-aba7-28d00d0472be-maas-api-tls\") pod \"maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:41:59.466469 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.466300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68d71df-1fe2-4592-aba7-28d00d0472be-openshift-service-ca\") pod \"maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:41:59.466469 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.466429    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qh4x\" (UniqueName: \"kubernetes.io/projected/d68d71df-1fe2-4592-aba7-28d00d0472be-kube-api-access-8qh4x\") pod \"maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:41:59.466980 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.466959    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68d71df-1fe2-4592-aba7-28d00d0472be-openshift-service-ca\") pod \"maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:41:59.468731 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.468713    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d68d71df-1fe2-4592-aba7-28d00d0472be-maas-api-tls\") pod \"maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:41:59.478290 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.478268    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qh4x\" (UniqueName: \"kubernetes.io/projected/d68d71df-1fe2-4592-aba7-28d00d0472be-kube-api-access-8qh4x\") pod \"maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:41:59.675345 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.675306    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:41:59.851338 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:41:59.851308    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"]
Jul 07 23:41:59.853340 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:41:59.853307    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68d71df_1fe2_4592_aba7_28d00d0472be.slice/crio-22b8742a428736a48181d1de7935052cce9b5e36ac99e4f8fcb98ad1afa147dc WatchSource:0}: Error finding container 22b8742a428736a48181d1de7935052cce9b5e36ac99e4f8fcb98ad1afa147dc: Status 404 returned error can't find the container with id 22b8742a428736a48181d1de7935052cce9b5e36ac99e4f8fcb98ad1afa147dc
Jul 07 23:42:00.118522 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:00.118439    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4" event={"ID":"d68d71df-1fe2-4592-aba7-28d00d0472be","Type":"ContainerStarted","Data":"22b8742a428736a48181d1de7935052cce9b5e36ac99e4f8fcb98ad1afa147dc"}
Jul 07 23:42:01.125217 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:01.125178    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4" event={"ID":"d68d71df-1fe2-4592-aba7-28d00d0472be","Type":"ContainerStarted","Data":"d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851"}
Jul 07 23:42:01.125671 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:01.125291    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:42:01.153957 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:01.153901    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4" podStartSLOduration=1.685145479 podStartE2EDuration="2.153881895s" totalImagesPullingTime="468.736416ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:42:01.152148587 +0000 UTC m=+1705.737717947" watchObservedRunningTime="2026-07-07 23:42:01.153881895 +0000 UTC m=+1705.739451255"
Jul 07 23:42:07.137005 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:07.136970    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:42:11.787218 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:11.787175    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"]
Jul 07 23:42:11.787736 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:11.787455    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4" podUID="d68d71df-1fe2-4592-aba7-28d00d0472be" containerName="maas-api" containerID="cri-o://d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851" gracePeriod=30
Jul 07 23:42:12.056242 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.056214    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:42:12.078352 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.078313    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d68d71df-1fe2-4592-aba7-28d00d0472be-maas-api-tls\") pod \"d68d71df-1fe2-4592-aba7-28d00d0472be\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") "
Jul 07 23:42:12.078584 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.078454    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68d71df-1fe2-4592-aba7-28d00d0472be-openshift-service-ca\") pod \"d68d71df-1fe2-4592-aba7-28d00d0472be\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") "
Jul 07 23:42:12.078584 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.078534    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qh4x\" (UniqueName: \"kubernetes.io/projected/d68d71df-1fe2-4592-aba7-28d00d0472be-kube-api-access-8qh4x\") pod \"d68d71df-1fe2-4592-aba7-28d00d0472be\" (UID: \"d68d71df-1fe2-4592-aba7-28d00d0472be\") "
Jul 07 23:42:12.079094 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.079062    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d68d71df-1fe2-4592-aba7-28d00d0472be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d68d71df-1fe2-4592-aba7-28d00d0472be" (UID: "d68d71df-1fe2-4592-aba7-28d00d0472be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:42:12.080902 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.080872    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d68d71df-1fe2-4592-aba7-28d00d0472be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d68d71df-1fe2-4592-aba7-28d00d0472be" (UID: "d68d71df-1fe2-4592-aba7-28d00d0472be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:42:12.081179 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.081159    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d68d71df-1fe2-4592-aba7-28d00d0472be-kube-api-access-8qh4x" (OuterVolumeSpecName: "kube-api-access-8qh4x") pod "d68d71df-1fe2-4592-aba7-28d00d0472be" (UID: "d68d71df-1fe2-4592-aba7-28d00d0472be"). InnerVolumeSpecName "kube-api-access-8qh4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:42:12.179345 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.179312    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qh4x\" (UniqueName: \"kubernetes.io/projected/d68d71df-1fe2-4592-aba7-28d00d0472be-kube-api-access-8qh4x\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:12.179345 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.179345    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d68d71df-1fe2-4592-aba7-28d00d0472be-maas-api-tls\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:12.179345 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.179358    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68d71df-1fe2-4592-aba7-28d00d0472be-openshift-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:12.184311 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.184276    2578 generic.go:356] "Generic (PLEG): container finished" podID="d68d71df-1fe2-4592-aba7-28d00d0472be" containerID="d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851" exitCode=0
Jul 07 23:42:12.184479 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.184324    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"
Jul 07 23:42:12.184479 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.184359    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4" event={"ID":"d68d71df-1fe2-4592-aba7-28d00d0472be","Type":"ContainerDied","Data":"d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851"}
Jul 07 23:42:12.184479 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.184402    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4" event={"ID":"d68d71df-1fe2-4592-aba7-28d00d0472be","Type":"ContainerDied","Data":"22b8742a428736a48181d1de7935052cce9b5e36ac99e4f8fcb98ad1afa147dc"}
Jul 07 23:42:12.184479 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.184431    2578 scope.go:117] "RemoveContainer" containerID="d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851"
Jul 07 23:42:12.186181 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.186155    2578 generic.go:356] "Generic (PLEG): container finished" podID="440142f5-70f0-4c77-8d99-06288f43255e" containerID="acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d" exitCode=0
Jul 07 23:42:12.186289 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.186211    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" event={"ID":"440142f5-70f0-4c77-8d99-06288f43255e","Type":"ContainerDied","Data":"acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d"}
Jul 07 23:42:12.186691 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.186671    2578 scope.go:117] "RemoveContainer" containerID="acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d"
Jul 07 23:42:12.187816 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.187786    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:42:12.187913 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.187876    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:42:12.187962 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.187912    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:42:12.194149 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.194130    2578 scope.go:117] "RemoveContainer" containerID="d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851"
Jul 07 23:42:12.194406 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:42:12.194387    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851\": container with ID starting with d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851 not found: ID does not exist" containerID="d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851"
Jul 07 23:42:12.194462 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.194415    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851"} err="failed to get container status \"d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851\": rpc error: code = NotFound desc = could not find container \"d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851\": container with ID starting with d1a63107269f82ba767b09aacd6964269d3899dfa78f2dd5d221a037e2b44851 not found: ID does not exist"
Jul 07 23:42:12.224161 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.224133    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"]
Jul 07 23:42:12.227905 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:12.227866    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27a7c412-568b7c7c-ps7z4"]
Jul 07 23:42:13.107947 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:13.107897    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:42:13.198432 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:13.198354    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" event={"ID":"440142f5-70f0-4c77-8d99-06288f43255e","Type":"ContainerStarted","Data":"54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8"}
Jul 07 23:42:13.491820 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:13.491784    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:42:13.492992 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:13.492954    2578 patch_prober.go:28] interesting pod/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 07 23:42:13.493119 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:13.493018    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" podUID="440142f5-70f0-4c77-8d99-06288f43255e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 07 23:42:13.963415 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:13.963379    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d68d71df-1fe2-4592-aba7-28d00d0472be" path="/var/lib/kubelet/pods/d68d71df-1fe2-4592-aba7-28d00d0472be/volumes"
Jul 07 23:42:14.493589 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:14.492386    2578 patch_prober.go:28] interesting pod/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 07 23:42:14.493589 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:14.492446    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" podUID="440142f5-70f0-4c77-8d99-06288f43255e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 07 23:42:15.491633 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:15.491596    2578 patch_prober.go:28] interesting pod/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 07 23:42:15.491920 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:15.491891    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" podUID="440142f5-70f0-4c77-8d99-06288f43255e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 07 23:42:16.491698 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:16.491654    2578 patch_prober.go:28] interesting pod/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 07 23:42:16.492231 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:16.491724    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" podUID="440142f5-70f0-4c77-8d99-06288f43255e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 07 23:42:17.496159 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:17.496127    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:42:17.496620 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:17.496414    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:42:17.496967 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:17.496953    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:42:18.225249 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:18.225213    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"]
Jul 07 23:42:18.227429 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:18.227399    2578 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" secret="" err="secret \"e2e-mt-27a7c412-openshift-default-dockercfg-w22nf\" not found"
Jul 07 23:42:19.232721 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:19.232651    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" podUID="440142f5-70f0-4c77-8d99-06288f43255e" containerName="istio-proxy" containerID="cri-o://54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8" gracePeriod=30
Jul 07 23:42:24.494639 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.494548    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:42:24.593480 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.593360    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-socket\") pod \"440142f5-70f0-4c77-8d99-06288f43255e\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") "
Jul 07 23:42:24.593480 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.593419    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-certs\") pod \"440142f5-70f0-4c77-8d99-06288f43255e\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") "
Jul 07 23:42:24.593480 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.593449    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-data\") pod \"440142f5-70f0-4c77-8d99-06288f43255e\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") "
Jul 07 23:42:24.593480 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.593469    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-envoy\") pod \"440142f5-70f0-4c77-8d99-06288f43255e\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") "
Jul 07 23:42:24.593480 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.593489    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/440142f5-70f0-4c77-8d99-06288f43255e-istio-podinfo\") pod \"440142f5-70f0-4c77-8d99-06288f43255e\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") "
Jul 07 23:42:24.593911 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.593508    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-istio-token\") pod \"440142f5-70f0-4c77-8d99-06288f43255e\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") "
Jul 07 23:42:24.593911 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.593528    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-credential-socket\") pod \"440142f5-70f0-4c77-8d99-06288f43255e\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") "
Jul 07 23:42:24.593911 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.593686    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/440142f5-70f0-4c77-8d99-06288f43255e-istiod-ca-cert\") pod \"440142f5-70f0-4c77-8d99-06288f43255e\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") "
Jul 07 23:42:24.593911 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.593691    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "440142f5-70f0-4c77-8d99-06288f43255e" (UID: "440142f5-70f0-4c77-8d99-06288f43255e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:42:24.593911 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.593730    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mtmv\" (UniqueName: \"kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-kube-api-access-5mtmv\") pod \"440142f5-70f0-4c77-8d99-06288f43255e\" (UID: \"440142f5-70f0-4c77-8d99-06288f43255e\") "
Jul 07 23:42:24.593911 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.593751    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "440142f5-70f0-4c77-8d99-06288f43255e" (UID: "440142f5-70f0-4c77-8d99-06288f43255e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:42:24.594242 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.594048    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:24.594242 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.594063    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/440142f5-70f0-4c77-8d99-06288f43255e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "440142f5-70f0-4c77-8d99-06288f43255e" (UID: "440142f5-70f0-4c77-8d99-06288f43255e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:42:24.594242 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.594070    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-workload-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:24.594400 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.594303    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "440142f5-70f0-4c77-8d99-06288f43255e" (UID: "440142f5-70f0-4c77-8d99-06288f43255e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:42:24.594650 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.594625    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-data" (OuterVolumeSpecName: "istio-data") pod "440142f5-70f0-4c77-8d99-06288f43255e" (UID: "440142f5-70f0-4c77-8d99-06288f43255e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:42:24.595825 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.595794    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/440142f5-70f0-4c77-8d99-06288f43255e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "440142f5-70f0-4c77-8d99-06288f43255e" (UID: "440142f5-70f0-4c77-8d99-06288f43255e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 23:42:24.595917 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.595855    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "440142f5-70f0-4c77-8d99-06288f43255e" (UID: "440142f5-70f0-4c77-8d99-06288f43255e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:42:24.595917 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.595881    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-istio-token" (OuterVolumeSpecName: "istio-token") pod "440142f5-70f0-4c77-8d99-06288f43255e" (UID: "440142f5-70f0-4c77-8d99-06288f43255e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:42:24.595997 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.595944    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-kube-api-access-5mtmv" (OuterVolumeSpecName: "kube-api-access-5mtmv") pod "440142f5-70f0-4c77-8d99-06288f43255e" (UID: "440142f5-70f0-4c77-8d99-06288f43255e"). InnerVolumeSpecName "kube-api-access-5mtmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:42:24.695163 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.695128    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-data\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:24.695163 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.695158    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-istio-envoy\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:24.695163 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.695169    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/440142f5-70f0-4c77-8d99-06288f43255e-istio-podinfo\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:24.695163 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.695177    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:24.695447 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.695184    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/440142f5-70f0-4c77-8d99-06288f43255e-credential-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:24.695447 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.695195    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/440142f5-70f0-4c77-8d99-06288f43255e-istiod-ca-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:24.695447 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:24.695203    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mtmv\" (UniqueName: \"kubernetes.io/projected/440142f5-70f0-4c77-8d99-06288f43255e-kube-api-access-5mtmv\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:42:25.266857 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.266819    2578 generic.go:356] "Generic (PLEG): container finished" podID="440142f5-70f0-4c77-8d99-06288f43255e" containerID="54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8" exitCode=0
Jul 07 23:42:25.267064 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.266868    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"
Jul 07 23:42:25.267064 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.266902    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" event={"ID":"440142f5-70f0-4c77-8d99-06288f43255e","Type":"ContainerDied","Data":"54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8"}
Jul 07 23:42:25.267064 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.266953    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd" event={"ID":"440142f5-70f0-4c77-8d99-06288f43255e","Type":"ContainerDied","Data":"06fa5fe237cc74f218d882797c026b9990901707a265217aa71076b69a277dd1"}
Jul 07 23:42:25.267064 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.266979    2578 scope.go:117] "RemoveContainer" containerID="54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8"
Jul 07 23:42:25.276100 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.276077    2578 scope.go:117] "RemoveContainer" containerID="acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d"
Jul 07 23:42:25.284363 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.284344    2578 scope.go:117] "RemoveContainer" containerID="54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8"
Jul 07 23:42:25.284665 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:42:25.284642    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8\": container with ID starting with 54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8 not found: ID does not exist" containerID="54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8"
Jul 07 23:42:25.284723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.284676    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8"} err="failed to get container status \"54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8\": rpc error: code = NotFound desc = could not find container \"54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8\": container with ID starting with 54b029b54d320d88fce352dbcfe369e24a30340669c37d304670f3ea64a2cce8 not found: ID does not exist"
Jul 07 23:42:25.284723 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.284694    2578 scope.go:117] "RemoveContainer" containerID="acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d"
Jul 07 23:42:25.284967 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:42:25.284949    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d\": container with ID starting with acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d not found: ID does not exist" containerID="acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d"
Jul 07 23:42:25.285016 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.284973    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d"} err="failed to get container status \"acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d\": rpc error: code = NotFound desc = could not find container \"acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d\": container with ID starting with acdd6a09c9b170ff83498a6fc76e9b7d88dd72539dffc8290df38b55fada512d not found: ID does not exist"
Jul 07 23:42:25.289735 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.289707    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"]
Jul 07 23:42:25.293570 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.293535    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-27a7c412-openshift-default-896ddffcf-njtpd"]
Jul 07 23:42:25.962987 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:25.962956    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="440142f5-70f0-4c77-8d99-06288f43255e" path="/var/lib/kubelet/pods/440142f5-70f0-4c77-8d99-06288f43255e/volumes"
Jul 07 23:42:38.283717 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.283654    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"]
Jul 07 23:42:38.284380 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.284263    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="440142f5-70f0-4c77-8d99-06288f43255e" containerName="istio-proxy"
Jul 07 23:42:38.284380 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.284283    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="440142f5-70f0-4c77-8d99-06288f43255e" containerName="istio-proxy"
Jul 07 23:42:38.284380 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.284306    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="440142f5-70f0-4c77-8d99-06288f43255e" containerName="istio-proxy"
Jul 07 23:42:38.284380 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.284314    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="440142f5-70f0-4c77-8d99-06288f43255e" containerName="istio-proxy"
Jul 07 23:42:38.284380 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.284335    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68d71df-1fe2-4592-aba7-28d00d0472be" containerName="maas-api"
Jul 07 23:42:38.284380 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.284345    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68d71df-1fe2-4592-aba7-28d00d0472be" containerName="maas-api"
Jul 07 23:42:38.284747 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.284453    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d68d71df-1fe2-4592-aba7-28d00d0472be" containerName="maas-api"
Jul 07 23:42:38.284747 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.284468    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="440142f5-70f0-4c77-8d99-06288f43255e" containerName="istio-proxy"
Jul 07 23:42:38.287756 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.287739    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.290219 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.290198    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-dfd300c4-openshift-default-dockercfg-26fc7\""
Jul 07 23:42:38.299752 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.299726    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"]
Jul 07 23:42:38.424745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.424695    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-token\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.424910 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.424763    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istiod-ca-cert\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.424910 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.424791    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47bl7\" (UniqueName: \"kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-kube-api-access-47bl7\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.424910 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.424824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-podinfo\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.424910 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.424861    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-certs\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.424910 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.424886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-envoy\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.425128 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.424954    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-credential-socket\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.425164 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.425136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-data\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.425207 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.425184    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-socket\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526261 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-data\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526261 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526265    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-socket\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526535 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526393    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-token\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526535 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istiod-ca-cert\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526535 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47bl7\" (UniqueName: \"kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-kube-api-access-47bl7\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526535 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526471    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-podinfo\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526535 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526502    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-certs\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526535 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526529    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-envoy\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526873 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526608    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-credential-socket\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526873 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526748    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-data\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526873 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526792    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-socket\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.526983 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526920    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-certs\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.527019 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.526987    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-credential-socket\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.527205 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.527185    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istiod-ca-cert\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.528979 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.528950    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-envoy\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.529139 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.529121    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-podinfo\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.535452 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.535395    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47bl7\" (UniqueName: \"kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-kube-api-access-47bl7\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.535550 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.535510    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-token\") pod \"e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") " pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.599761 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.599724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:38.750030 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.749998    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"]
Jul 07 23:42:38.754047 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:42:38.754014    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2baaf1de_3ec5_4ec5_9a20_c875aeb2ea3d.slice/crio-c89facfa51f6f380257612b07a86a7688b51aa0d9746c156cfe6ee803bfde768 WatchSource:0}: Error finding container c89facfa51f6f380257612b07a86a7688b51aa0d9746c156cfe6ee803bfde768: Status 404 returned error can't find the container with id c89facfa51f6f380257612b07a86a7688b51aa0d9746c156cfe6ee803bfde768
Jul 07 23:42:38.756140 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.756107    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:42:38.756231 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.756174    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:42:38.756231 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:38.756204    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:42:39.336672 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:39.336635    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" event={"ID":"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d","Type":"ContainerStarted","Data":"09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c"}
Jul 07 23:42:39.337048 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:39.336692    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" event={"ID":"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d","Type":"ContainerStarted","Data":"c89facfa51f6f380257612b07a86a7688b51aa0d9746c156cfe6ee803bfde768"}
Jul 07 23:42:39.360495 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:39.360441    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" podStartSLOduration=1.360426031 podStartE2EDuration="1.360426031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:42:39.357709224 +0000 UTC m=+1743.943278599" watchObservedRunningTime="2026-07-07 23:42:39.360426031 +0000 UTC m=+1743.945995461"
Jul 07 23:42:39.600454 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:39.600367    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:39.605313 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:39.605276    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:40.341533 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:40.341498    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:40.342533 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:40.342510    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:41.314896 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.314860    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"]
Jul 07 23:42:41.315362 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.315346    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="440142f5-70f0-4c77-8d99-06288f43255e" containerName="istio-proxy"
Jul 07 23:42:41.318781 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.318759    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:41.321681 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.321655    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-dfd300c4-serving-cert\""
Jul 07 23:42:41.331534 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.331504    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"]
Jul 07 23:42:41.452244 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.452204    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwqwd\" (UniqueName: \"kubernetes.io/projected/792a9c98-771a-4c7f-a2bb-3ea617305727-kube-api-access-rwqwd\") pod \"maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:41.452772 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.452277    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/792a9c98-771a-4c7f-a2bb-3ea617305727-openshift-service-ca\") pod \"maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:41.452772 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.452306    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/792a9c98-771a-4c7f-a2bb-3ea617305727-maas-api-tls\") pod \"maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:41.553525 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.553484    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwqwd\" (UniqueName: \"kubernetes.io/projected/792a9c98-771a-4c7f-a2bb-3ea617305727-kube-api-access-rwqwd\") pod \"maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:41.553769 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.553550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/792a9c98-771a-4c7f-a2bb-3ea617305727-openshift-service-ca\") pod \"maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:41.553769 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.553612    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/792a9c98-771a-4c7f-a2bb-3ea617305727-maas-api-tls\") pod \"maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:41.553769 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:42:41.553733    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-serving-cert: secret "maas-api-e2e-mt-dfd300c4-serving-cert" not found
Jul 07 23:42:41.553930 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:42:41.553802    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/792a9c98-771a-4c7f-a2bb-3ea617305727-maas-api-tls podName:792a9c98-771a-4c7f-a2bb-3ea617305727 nodeName:}" failed. No retries permitted until 2026-07-07 23:42:42.053784597 +0000 UTC m=+1746.639353938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/792a9c98-771a-4c7f-a2bb-3ea617305727-maas-api-tls") pod "maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx" (UID: "792a9c98-771a-4c7f-a2bb-3ea617305727") : secret "maas-api-e2e-mt-dfd300c4-serving-cert" not found
Jul 07 23:42:41.554263 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.554240    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/792a9c98-771a-4c7f-a2bb-3ea617305727-openshift-service-ca\") pod \"maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:41.562699 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:41.562672    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwqwd\" (UniqueName: \"kubernetes.io/projected/792a9c98-771a-4c7f-a2bb-3ea617305727-kube-api-access-rwqwd\") pod \"maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:42.059238 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:42.059197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/792a9c98-771a-4c7f-a2bb-3ea617305727-maas-api-tls\") pod \"maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:42.061778 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:42.061753    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/792a9c98-771a-4c7f-a2bb-3ea617305727-maas-api-tls\") pod \"maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:42.232998 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:42.232962    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:42.375674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:42.375650    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"]
Jul 07 23:42:42.377481 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:42:42.377451    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod792a9c98_771a_4c7f_a2bb_3ea617305727.slice/crio-7ea00db77b7cc1ab08da15a956525aa17331842bc35c95d7c8d01efce5c946fc WatchSource:0}: Error finding container 7ea00db77b7cc1ab08da15a956525aa17331842bc35c95d7c8d01efce5c946fc: Status 404 returned error can't find the container with id 7ea00db77b7cc1ab08da15a956525aa17331842bc35c95d7c8d01efce5c946fc
Jul 07 23:42:43.359232 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:43.359197    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx" event={"ID":"792a9c98-771a-4c7f-a2bb-3ea617305727","Type":"ContainerStarted","Data":"68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da"}
Jul 07 23:42:43.359707 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:43.359263    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:43.359707 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:43.359297    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx" event={"ID":"792a9c98-771a-4c7f-a2bb-3ea617305727","Type":"ContainerStarted","Data":"7ea00db77b7cc1ab08da15a956525aa17331842bc35c95d7c8d01efce5c946fc"}
Jul 07 23:42:43.377282 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:43.377201    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx" podStartSLOduration=1.946639256 podStartE2EDuration="2.377184663s" totalImagesPullingTime="430.545407ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:42:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:42:43.376913509 +0000 UTC m=+1747.962482863" watchObservedRunningTime="2026-07-07 23:42:43.377184663 +0000 UTC m=+1747.962754023"
Jul 07 23:42:49.371041 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:49.371009    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:42:54.429079 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:54.429039    2578 generic.go:356] "Generic (PLEG): container finished" podID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerID="09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c" exitCode=0
Jul 07 23:42:54.429521 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:54.429117    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" event={"ID":"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d","Type":"ContainerDied","Data":"09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c"}
Jul 07 23:42:54.429626 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:54.429609    2578 scope.go:117] "RemoveContainer" containerID="09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c"
Jul 07 23:42:54.430645 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:54.430615    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:42:54.430751 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:54.430704    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:42:54.430815 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:54.430754    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:42:55.341485 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:55.341425    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:55.435545 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:55.435504    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" event={"ID":"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d","Type":"ContainerStarted","Data":"3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990"}
Jul 07 23:42:55.601749 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:55.601655    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:55.602257 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:55.602230    2578 patch_prober.go:28] interesting pod/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 07 23:42:55.602337 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:55.602312    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 07 23:42:56.600935 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:56.600880    2578 patch_prober.go:28] interesting pod/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 07 23:42:56.601475 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:56.600971    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 07 23:42:57.600256 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:57.600212    2578 patch_prober.go:28] interesting pod/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 07 23:42:57.600603 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:57.600552    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 07 23:42:58.600731 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:58.600687    2578 patch_prober.go:28] interesting pod/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 07 23:42:58.601368 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:58.601325    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 07 23:42:59.604139 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:59.604106    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:59.604635 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:59.604526    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:42:59.605324 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:42:59.605305    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:43:28.711995 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:28.711903    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"]
Jul 07 23:43:28.712766 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:28.712727    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx" podUID="792a9c98-771a-4c7f-a2bb-3ea617305727" containerName="maas-api" containerID="cri-o://68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da" gracePeriod=30
Jul 07 23:43:28.973004 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:28.972926    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:43:29.098292 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.098260    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwqwd\" (UniqueName: \"kubernetes.io/projected/792a9c98-771a-4c7f-a2bb-3ea617305727-kube-api-access-rwqwd\") pod \"792a9c98-771a-4c7f-a2bb-3ea617305727\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") "
Jul 07 23:43:29.098501 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.098308    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/792a9c98-771a-4c7f-a2bb-3ea617305727-maas-api-tls\") pod \"792a9c98-771a-4c7f-a2bb-3ea617305727\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") "
Jul 07 23:43:29.098501 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.098335    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/792a9c98-771a-4c7f-a2bb-3ea617305727-openshift-service-ca\") pod \"792a9c98-771a-4c7f-a2bb-3ea617305727\" (UID: \"792a9c98-771a-4c7f-a2bb-3ea617305727\") "
Jul 07 23:43:29.098778 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.098750    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/792a9c98-771a-4c7f-a2bb-3ea617305727-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "792a9c98-771a-4c7f-a2bb-3ea617305727" (UID: "792a9c98-771a-4c7f-a2bb-3ea617305727"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:43:29.100585 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.100545    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/792a9c98-771a-4c7f-a2bb-3ea617305727-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "792a9c98-771a-4c7f-a2bb-3ea617305727" (UID: "792a9c98-771a-4c7f-a2bb-3ea617305727"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:43:29.100670 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.100592    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/792a9c98-771a-4c7f-a2bb-3ea617305727-kube-api-access-rwqwd" (OuterVolumeSpecName: "kube-api-access-rwqwd") pod "792a9c98-771a-4c7f-a2bb-3ea617305727" (UID: "792a9c98-771a-4c7f-a2bb-3ea617305727"). InnerVolumeSpecName "kube-api-access-rwqwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:43:29.199259 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.199223    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwqwd\" (UniqueName: \"kubernetes.io/projected/792a9c98-771a-4c7f-a2bb-3ea617305727-kube-api-access-rwqwd\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:43:29.199259 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.199253    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/792a9c98-771a-4c7f-a2bb-3ea617305727-maas-api-tls\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:43:29.199259 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.199262    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/792a9c98-771a-4c7f-a2bb-3ea617305727-openshift-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:43:29.601132 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.601101    2578 generic.go:356] "Generic (PLEG): container finished" podID="792a9c98-771a-4c7f-a2bb-3ea617305727" containerID="68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da" exitCode=0
Jul 07 23:43:29.601348 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.601145    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"
Jul 07 23:43:29.601348 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.601187    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx" event={"ID":"792a9c98-771a-4c7f-a2bb-3ea617305727","Type":"ContainerDied","Data":"68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da"}
Jul 07 23:43:29.601348 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.601245    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx" event={"ID":"792a9c98-771a-4c7f-a2bb-3ea617305727","Type":"ContainerDied","Data":"7ea00db77b7cc1ab08da15a956525aa17331842bc35c95d7c8d01efce5c946fc"}
Jul 07 23:43:29.601348 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.601287    2578 scope.go:117] "RemoveContainer" containerID="68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da"
Jul 07 23:43:29.610232 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.610213    2578 scope.go:117] "RemoveContainer" containerID="68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da"
Jul 07 23:43:29.610501 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:43:29.610478    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da\": container with ID starting with 68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da not found: ID does not exist" containerID="68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da"
Jul 07 23:43:29.610580 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.610510    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da"} err="failed to get container status \"68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da\": rpc error: code = NotFound desc = could not find container \"68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da\": container with ID starting with 68a130630fbca409dcf50f12dffd635bbb8f83317e2a11bf1e8375bdf4bd38da not found: ID does not exist"
Jul 07 23:43:29.624057 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.624029    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"]
Jul 07 23:43:29.629749 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.629726    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd300c4-7b569d8d5b-7zdpx"]
Jul 07 23:43:29.963507 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:43:29.963473    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="792a9c98-771a-4c7f-a2bb-3ea617305727" path="/var/lib/kubelet/pods/792a9c98-771a-4c7f-a2bb-3ea617305727/volumes"
Jul 07 23:44:22.599545 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:22.599506    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"]
Jul 07 23:44:22.600169 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:22.599788    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerName="istio-proxy" containerID="cri-o://3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990" gracePeriod=30
Jul 07 23:44:27.851768 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.851743    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:44:27.896482 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.896444    2578 generic.go:356] "Generic (PLEG): container finished" podID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerID="3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990" exitCode=0
Jul 07 23:44:27.896769 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.896489    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" event={"ID":"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d","Type":"ContainerDied","Data":"3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990"}
Jul 07 23:44:27.896769 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.896508    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"
Jul 07 23:44:27.896769 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.896574    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8" event={"ID":"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d","Type":"ContainerDied","Data":"c89facfa51f6f380257612b07a86a7688b51aa0d9746c156cfe6ee803bfde768"}
Jul 07 23:44:27.896769 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.896591    2578 scope.go:117] "RemoveContainer" containerID="3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990"
Jul 07 23:44:27.905784 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.905760    2578 scope.go:117] "RemoveContainer" containerID="09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c"
Jul 07 23:44:27.913687 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.913666    2578 scope.go:117] "RemoveContainer" containerID="3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990"
Jul 07 23:44:27.913954 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:44:27.913935    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990\": container with ID starting with 3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990 not found: ID does not exist" containerID="3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990"
Jul 07 23:44:27.914014 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.913967    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990"} err="failed to get container status \"3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990\": rpc error: code = NotFound desc = could not find container \"3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990\": container with ID starting with 3a1c2e213093e7900984d5ae054f38eb3a59c36ebde16d81fc68bdfccf7f3990 not found: ID does not exist"
Jul 07 23:44:27.914014 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.913988    2578 scope.go:117] "RemoveContainer" containerID="09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c"
Jul 07 23:44:27.914231 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:44:27.914209    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c\": container with ID starting with 09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c not found: ID does not exist" containerID="09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c"
Jul 07 23:44:27.914278 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.914238    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c"} err="failed to get container status \"09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c\": rpc error: code = NotFound desc = could not find container \"09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c\": container with ID starting with 09d9259fc3c3c35f36edc276b2d4007c498881a39256f25610899e0ea3a1828c not found: ID does not exist"
Jul 07 23:44:27.946289 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946255    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-podinfo\") pod \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") "
Jul 07 23:44:27.946471 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946299    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-credential-socket\") pod \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") "
Jul 07 23:44:27.946471 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946320    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47bl7\" (UniqueName: \"kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-kube-api-access-47bl7\") pod \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") "
Jul 07 23:44:27.946471 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946436    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istiod-ca-cert\") pod \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") "
Jul 07 23:44:27.946681 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946482    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-socket\") pod \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") "
Jul 07 23:44:27.946681 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946516    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-envoy\") pod \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") "
Jul 07 23:44:27.946681 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946544    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-token\") pod \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") "
Jul 07 23:44:27.946681 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946591    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-data\") pod \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") "
Jul 07 23:44:27.946681 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946601    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" (UID: "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:44:27.946681 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946650    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-certs\") pod \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\" (UID: \"2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d\") "
Jul 07 23:44:27.947002 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946769    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" (UID: "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:44:27.947002 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946828    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" (UID: "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:44:27.947002 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.946993    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" (UID: "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:44:27.947170 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.947054    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istiod-ca-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:44:27.947170 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.947073    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:44:27.947170 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.947087    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-credential-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:44:27.947606 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.947583    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-data" (OuterVolumeSpecName: "istio-data") pod "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" (UID: "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:44:27.948597 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.948549    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-kube-api-access-47bl7" (OuterVolumeSpecName: "kube-api-access-47bl7") pod "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" (UID: "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d"). InnerVolumeSpecName "kube-api-access-47bl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:44:27.948773 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.948645    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-token" (OuterVolumeSpecName: "istio-token") pod "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" (UID: "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:44:27.948910 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.948888    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" (UID: "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 23:44:27.948955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:27.948922    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" (UID: "2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:44:28.047595 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:28.047510    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-envoy\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:44:28.047595 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:28.047592    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:44:28.047595 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:28.047602    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-data\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:44:28.047595 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:28.047610    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-workload-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:44:28.047595 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:28.047619    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-istio-podinfo\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:44:28.047913 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:28.047648    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47bl7\" (UniqueName: \"kubernetes.io/projected/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d-kube-api-access-47bl7\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:44:28.214514 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:28.214422    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"]
Jul 07 23:44:28.220011 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:28.219984    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-dfd300c4-openshift-default-cc94cdd7f-xbfh8"]
Jul 07 23:44:29.963358 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:44:29.963322    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" path="/var/lib/kubelet/pods/2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d/volumes"
Jul 07 23:45:00.142873 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.142784    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"]
Jul 07 23:45:00.143382 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.143167    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="792a9c98-771a-4c7f-a2bb-3ea617305727" containerName="maas-api"
Jul 07 23:45:00.143382 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.143178    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="792a9c98-771a-4c7f-a2bb-3ea617305727" containerName="maas-api"
Jul 07 23:45:00.143382 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.143201    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerName="istio-proxy"
Jul 07 23:45:00.143382 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.143207    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerName="istio-proxy"
Jul 07 23:45:00.143382 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.143214    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerName="istio-proxy"
Jul 07 23:45:00.143382 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.143221    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerName="istio-proxy"
Jul 07 23:45:00.143382 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.143274    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="792a9c98-771a-4c7f-a2bb-3ea617305727" containerName="maas-api"
Jul 07 23:45:00.143382 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.143283    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerName="istio-proxy"
Jul 07 23:45:00.143382 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.143290    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2baaf1de-3ec5-4ec5-9a20-c875aeb2ea3d" containerName="istio-proxy"
Jul 07 23:45:00.146422 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.146402    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"
Jul 07 23:45:00.152404 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.152371    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"]
Jul 07 23:45:00.249878 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.249835    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfkq7\" (UniqueName: \"kubernetes.io/projected/57f021cf-b2d0-4848-b14b-e99520e18d81-kube-api-access-bfkq7\") pod \"maas-api-key-cleanup-29724465-qxpwd\" (UID: \"57f021cf-b2d0-4848-b14b-e99520e18d81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"
Jul 07 23:45:00.351071 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.351020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfkq7\" (UniqueName: \"kubernetes.io/projected/57f021cf-b2d0-4848-b14b-e99520e18d81-kube-api-access-bfkq7\") pod \"maas-api-key-cleanup-29724465-qxpwd\" (UID: \"57f021cf-b2d0-4848-b14b-e99520e18d81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"
Jul 07 23:45:00.360002 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.359964    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfkq7\" (UniqueName: \"kubernetes.io/projected/57f021cf-b2d0-4848-b14b-e99520e18d81-kube-api-access-bfkq7\") pod \"maas-api-key-cleanup-29724465-qxpwd\" (UID: \"57f021cf-b2d0-4848-b14b-e99520e18d81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"
Jul 07 23:45:00.457679 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.457527    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"
Jul 07 23:45:00.593011 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:00.592979    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"]
Jul 07 23:45:00.594980 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:45:00.594945    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f021cf_b2d0_4848_b14b_e99520e18d81.slice/crio-5d21139df56dcc6afe82684acbc6c8690f8cc6ff45929f44900395d681bb1ed3 WatchSource:0}: Error finding container 5d21139df56dcc6afe82684acbc6c8690f8cc6ff45929f44900395d681bb1ed3: Status 404 returned error can't find the container with id 5d21139df56dcc6afe82684acbc6c8690f8cc6ff45929f44900395d681bb1ed3
Jul 07 23:45:01.063648 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:01.063533    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd" event={"ID":"57f021cf-b2d0-4848-b14b-e99520e18d81","Type":"ContainerStarted","Data":"083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf"}
Jul 07 23:45:01.063648 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:01.063610    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd" event={"ID":"57f021cf-b2d0-4848-b14b-e99520e18d81","Type":"ContainerStarted","Data":"5d21139df56dcc6afe82684acbc6c8690f8cc6ff45929f44900395d681bb1ed3"}
Jul 07 23:45:01.082151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:01.081793    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd" podStartSLOduration=1.081772138 podStartE2EDuration="1.081772138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:45:01.079521408 +0000 UTC m=+1885.665090768" watchObservedRunningTime="2026-07-07 23:45:01.081772138 +0000 UTC m=+1885.667341502"
Jul 07 23:45:17.866316 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.866276    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"]
Jul 07 23:45:17.870232 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.870210    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:17.873126 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.873106    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-04f579-openshift-default-dockercfg-dmhmp\""
Jul 07 23:45:17.879994 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.879965    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"]
Jul 07 23:45:17.916610 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.916538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2299d28c-4d51-4adf-8878-a801c16f43a7-istiod-ca-cert\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:17.916793 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.916641    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-credential-socket\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:17.916793 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.916677    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-certs\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:17.916793 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.916700    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-data\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:17.916793 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.916731    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-envoy\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:17.916937 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.916833    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-token\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:17.916937 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.916875    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-podinfo\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:17.916937 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.916892    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf4jv\" (UniqueName: \"kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-kube-api-access-xf4jv\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:17.916937 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:17.916911    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-socket\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.018138 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.018096    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-token\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.018344 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.018149    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-podinfo\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.018344 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.018175    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4jv\" (UniqueName: \"kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-kube-api-access-xf4jv\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.018344 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.018204    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-socket\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.018344 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.018238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2299d28c-4d51-4adf-8878-a801c16f43a7-istiod-ca-cert\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.018344 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.018282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-credential-socket\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.018344 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.018327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-certs\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.018762 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.018363    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-data\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.018762 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.018408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-envoy\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.018874 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.018854    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-credential-socket\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.019049 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.019024    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-certs\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.019128 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.019106    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-socket\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.019187 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.019169    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2299d28c-4d51-4adf-8878-a801c16f43a7-istiod-ca-cert\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.019265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.019250    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-data\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.020771 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.020752    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-envoy\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.020972 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.020950    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-podinfo\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.027024 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.026991    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf4jv\" (UniqueName: \"kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-kube-api-access-xf4jv\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.027392 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.027371    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-token\") pod \"e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") " pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.183554 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.183504    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:18.344578 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.344534    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"]
Jul 07 23:45:18.346960 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:45:18.346935    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2299d28c_4d51_4adf_8878_a801c16f43a7.slice/crio-2ae02fae828360a866a0615a061a3b8a24f7a1a4294f421227e1e17e41813fbc WatchSource:0}: Error finding container 2ae02fae828360a866a0615a061a3b8a24f7a1a4294f421227e1e17e41813fbc: Status 404 returned error can't find the container with id 2ae02fae828360a866a0615a061a3b8a24f7a1a4294f421227e1e17e41813fbc
Jul 07 23:45:18.349151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.349116    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:18.349262 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.349187    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:18.349262 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:18.349237    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:19.151849 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:19.151810    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" event={"ID":"2299d28c-4d51-4adf-8878-a801c16f43a7","Type":"ContainerStarted","Data":"bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6"}
Jul 07 23:45:19.152283 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:19.151866    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" event={"ID":"2299d28c-4d51-4adf-8878-a801c16f43a7","Type":"ContainerStarted","Data":"2ae02fae828360a866a0615a061a3b8a24f7a1a4294f421227e1e17e41813fbc"}
Jul 07 23:45:19.183730 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:19.183683    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:19.188722 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:19.188694    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:19.194811 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:19.194764    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" podStartSLOduration=2.19474962 podStartE2EDuration="2.19474962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:45:19.194319186 +0000 UTC m=+1903.779888559" watchObservedRunningTime="2026-07-07 23:45:19.19474962 +0000 UTC m=+1903.780319049"
Jul 07 23:45:20.157289 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:20.157256    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:20.158310 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:20.158292    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:21.354731 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.354683    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"]
Jul 07 23:45:21.358239 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.358219    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:21.360828 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.360806    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-04f579-serving-cert\""
Jul 07 23:45:21.369486 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.369453    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"]
Jul 07 23:45:21.451302 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.451268    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c09431d5-b557-4883-9acd-69028260b1f7-maas-api-tls\") pod \"maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:21.451511 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.451320    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c09431d5-b557-4883-9acd-69028260b1f7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:21.451511 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.451371    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmtcb\" (UniqueName: \"kubernetes.io/projected/c09431d5-b557-4883-9acd-69028260b1f7-kube-api-access-zmtcb\") pod \"maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:21.552540 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.552499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmtcb\" (UniqueName: \"kubernetes.io/projected/c09431d5-b557-4883-9acd-69028260b1f7-kube-api-access-zmtcb\") pod \"maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:21.552773 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.552620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c09431d5-b557-4883-9acd-69028260b1f7-maas-api-tls\") pod \"maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:21.552773 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.552677    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c09431d5-b557-4883-9acd-69028260b1f7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:21.552870 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:45:21.552771    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-serving-cert: secret "maas-api-e2e-inf-a-04f579-serving-cert" not found
Jul 07 23:45:21.552870 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:45:21.552838    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c09431d5-b557-4883-9acd-69028260b1f7-maas-api-tls podName:c09431d5-b557-4883-9acd-69028260b1f7 nodeName:}" failed. No retries permitted until 2026-07-07 23:45:22.052820834 +0000 UTC m=+1906.638390173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c09431d5-b557-4883-9acd-69028260b1f7-maas-api-tls") pod "maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws" (UID: "c09431d5-b557-4883-9acd-69028260b1f7") : secret "maas-api-e2e-inf-a-04f579-serving-cert" not found
Jul 07 23:45:21.553423 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.553404    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c09431d5-b557-4883-9acd-69028260b1f7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:21.562128 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:21.562099    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmtcb\" (UniqueName: \"kubernetes.io/projected/c09431d5-b557-4883-9acd-69028260b1f7-kube-api-access-zmtcb\") pod \"maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:22.057616 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:22.057583    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c09431d5-b557-4883-9acd-69028260b1f7-maas-api-tls\") pod \"maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:22.060020 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:22.059994    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c09431d5-b557-4883-9acd-69028260b1f7-maas-api-tls\") pod \"maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:22.269853 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:22.269818    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:22.410892 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:22.410862    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"]
Jul 07 23:45:22.413555 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:45:22.413525    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc09431d5_b557_4883_9acd_69028260b1f7.slice/crio-2a917bb4990212ad50de331ce29d01cbe21f946795d26d00c7e9f011fe83bafa WatchSource:0}: Error finding container 2a917bb4990212ad50de331ce29d01cbe21f946795d26d00c7e9f011fe83bafa: Status 404 returned error can't find the container with id 2a917bb4990212ad50de331ce29d01cbe21f946795d26d00c7e9f011fe83bafa
Jul 07 23:45:22.415150 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:22.415132    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 23:45:23.179005 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:23.178971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws" event={"ID":"c09431d5-b557-4883-9acd-69028260b1f7","Type":"ContainerStarted","Data":"b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd"}
Jul 07 23:45:23.179225 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:23.179036    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws" event={"ID":"c09431d5-b557-4883-9acd-69028260b1f7","Type":"ContainerStarted","Data":"2a917bb4990212ad50de331ce29d01cbe21f946795d26d00c7e9f011fe83bafa"}
Jul 07 23:45:23.179225 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:23.179141    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:23.198151 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:23.198090    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws" podStartSLOduration=1.781492991 podStartE2EDuration="2.198070094s" totalImagesPullingTime="416.577103ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:45:23.197243062 +0000 UTC m=+1907.782812423" watchObservedRunningTime="2026-07-07 23:45:23.198070094 +0000 UTC m=+1907.783639459"
Jul 07 23:45:26.218121 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.218082    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"]
Jul 07 23:45:26.222808 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.222773    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.226765 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.226740    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c324fe-openshift-default-dockercfg-qds8l\""
Jul 07 23:45:26.245711 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.245673    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"]
Jul 07 23:45:26.297543 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.297506    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxmgt\" (UniqueName: \"kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-kube-api-access-lxmgt\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.297543 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.297551    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf2ee95-a138-46c5-aacd-3205079b20ae-istiod-ca-cert\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.297838 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.297603    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-envoy\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.297838 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.297649    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-podinfo\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.297838 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.297677    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-socket\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.297838 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.297694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-token\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.297838 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.297716    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-credential-socket\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.297838 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.297749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-certs\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.297838 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.297781    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-data\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.398728 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.398689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-data\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.398944 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.398827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxmgt\" (UniqueName: \"kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-kube-api-access-lxmgt\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.398944 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.398855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf2ee95-a138-46c5-aacd-3205079b20ae-istiod-ca-cert\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.398944 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.398876    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-envoy\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.398944 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.398893    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-podinfo\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.398944 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.398912    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-socket\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.398944 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.398932    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-token\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.399260 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.398968    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-credential-socket\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.399260 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.399014    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-certs\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.399260 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.399167    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-data\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.399415 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.399288    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-socket\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.399415 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.399362    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-certs\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.399575 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.399543    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-credential-socket\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.399645 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.399612    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf2ee95-a138-46c5-aacd-3205079b20ae-istiod-ca-cert\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.401278 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.401257    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-envoy\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.401519 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.401497    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-podinfo\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.414207 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.414176    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-token\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.418168 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.418132    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxmgt\" (UniqueName: \"kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-kube-api-access-lxmgt\") pod \"e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") " pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.533897 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.533802    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:26.683517 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.683488    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"]
Jul 07 23:45:26.685136 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:45:26.685107    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf2ee95_a138_46c5_aacd_3205079b20ae.slice/crio-ef3675c7780a48950c5ada5283e462ccf53dbd9b40632a083760c54b159e9bc3 WatchSource:0}: Error finding container ef3675c7780a48950c5ada5283e462ccf53dbd9b40632a083760c54b159e9bc3: Status 404 returned error can't find the container with id ef3675c7780a48950c5ada5283e462ccf53dbd9b40632a083760c54b159e9bc3
Jul 07 23:45:26.687904 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.687873    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:26.687982 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.687945    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:26.687982 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:26.687977    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:27.200035 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:27.199986    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" event={"ID":"1bf2ee95-a138-46c5-aacd-3205079b20ae","Type":"ContainerStarted","Data":"5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323"}
Jul 07 23:45:27.200218 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:27.200054    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" event={"ID":"1bf2ee95-a138-46c5-aacd-3205079b20ae","Type":"ContainerStarted","Data":"ef3675c7780a48950c5ada5283e462ccf53dbd9b40632a083760c54b159e9bc3"}
Jul 07 23:45:27.223246 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:27.223191    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" podStartSLOduration=1.223171677 podStartE2EDuration="1.223171677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:45:27.220610647 +0000 UTC m=+1911.806180007" watchObservedRunningTime="2026-07-07 23:45:27.223171677 +0000 UTC m=+1911.808741056"
Jul 07 23:45:27.534984 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:27.534876    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:27.540079 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:27.540048    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:28.204736 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:28.204705    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:28.205602 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:28.205582    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:29.191432 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.191401    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:45:29.193101 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.193075    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"]
Jul 07 23:45:29.196925 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.196905    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:29.199231 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.199210    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c324fe-serving-cert\""
Jul 07 23:45:29.212979 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.212941    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"]
Jul 07 23:45:29.225360 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.225307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfqbk\" (UniqueName: \"kubernetes.io/projected/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-kube-api-access-pfqbk\") pod \"maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:29.225552 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.225467    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-maas-api-tls\") pod \"maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:29.225552 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.225512    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:29.326840 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.326802    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-maas-api-tls\") pod \"maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:29.327174 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.327148    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:29.327370 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:45:29.327188    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-serving-cert: secret "maas-api-e2e-inf-b-c324fe-serving-cert" not found
Jul 07 23:45:29.327484 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:45:29.327461    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-maas-api-tls podName:2903a1e8-9fe4-41d0-8c16-83515e0c64bd nodeName:}" failed. No retries permitted until 2026-07-07 23:45:29.827432217 +0000 UTC m=+1914.413001562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-maas-api-tls") pod "maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558" (UID: "2903a1e8-9fe4-41d0-8c16-83515e0c64bd") : secret "maas-api-e2e-inf-b-c324fe-serving-cert" not found
Jul 07 23:45:29.327484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.327458    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfqbk\" (UniqueName: \"kubernetes.io/projected/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-kube-api-access-pfqbk\") pod \"maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:29.328036 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.328012    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:29.344355 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.344318    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfqbk\" (UniqueName: \"kubernetes.io/projected/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-kube-api-access-pfqbk\") pod \"maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:29.832527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.832489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-maas-api-tls\") pod \"maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:29.835193 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:29.835167    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-maas-api-tls\") pod \"maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:30.110490 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:30.110390    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:30.272939 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:30.272906    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"]
Jul 07 23:45:30.275201 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:45:30.275171    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2903a1e8_9fe4_41d0_8c16_83515e0c64bd.slice/crio-d9af356affe433f52d7ec89874fd390f1d38433b794140a878fecbf2d35f9c7f WatchSource:0}: Error finding container d9af356affe433f52d7ec89874fd390f1d38433b794140a878fecbf2d35f9c7f: Status 404 returned error can't find the container with id d9af356affe433f52d7ec89874fd390f1d38433b794140a878fecbf2d35f9c7f
Jul 07 23:45:31.231887 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:31.231855    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558" event={"ID":"2903a1e8-9fe4-41d0-8c16-83515e0c64bd","Type":"ContainerStarted","Data":"1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347"}
Jul 07 23:45:31.232081 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:31.231907    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558" event={"ID":"2903a1e8-9fe4-41d0-8c16-83515e0c64bd","Type":"ContainerStarted","Data":"d9af356affe433f52d7ec89874fd390f1d38433b794140a878fecbf2d35f9c7f"}
Jul 07 23:45:31.232081 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:31.232035    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:31.248572 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:31.248498    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558" podStartSLOduration=1.9261829499999998 podStartE2EDuration="2.248477338s" totalImagesPullingTime="322.294388ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:45:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:45:31.247953809 +0000 UTC m=+1915.833523172" watchObservedRunningTime="2026-07-07 23:45:31.248477338 +0000 UTC m=+1915.834046701"
Jul 07 23:45:36.202726 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:36.202676    2578 patch_prober.go:28] interesting pod/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 23:45:36.203222 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:36.202752    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 23:45:37.242431 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:37.242402    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:45:37.264925 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:37.264891    2578 generic.go:356] "Generic (PLEG): container finished" podID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerID="bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6" exitCode=0
Jul 07 23:45:37.265107 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:37.264963    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" event={"ID":"2299d28c-4d51-4adf-8878-a801c16f43a7","Type":"ContainerDied","Data":"bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6"}
Jul 07 23:45:37.265455 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:37.265427    2578 scope.go:117] "RemoveContainer" containerID="bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6"
Jul 07 23:45:37.266686 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:37.266655    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:37.267013 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:37.266725    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:37.267013 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:37.266759    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:38.273217 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:38.273185    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" event={"ID":"2299d28c-4d51-4adf-8878-a801c16f43a7","Type":"ContainerStarted","Data":"f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4"}
Jul 07 23:45:39.184747 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:39.184705    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:39.185076 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:39.184888    2578 patch_prober.go:28] interesting pod/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 07 23:45:39.185276 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:39.185246    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 07 23:45:40.184051 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:40.184007    2578 patch_prober.go:28] interesting pod/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 07 23:45:40.184666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:40.184633    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 07 23:45:41.184436 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:41.184392    2578 patch_prober.go:28] interesting pod/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 07 23:45:41.185081 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:41.185041    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 07 23:45:42.184088 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:42.184037    2578 patch_prober.go:28] interesting pod/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 07 23:45:42.184314 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:42.184155    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 07 23:45:43.188683 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:43.188650    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:43.189107 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:43.188983    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:43.189699 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:43.189680    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:45:43.754353 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:43.753292    2578 patch_prober.go:28] interesting pod/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": read tcp 10.133.0.2:60804->10.133.0.61:15021: read: connection reset by peer" start-of-body=
Jul 07 23:45:43.754353 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:43.753407    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": read tcp 10.133.0.2:60804->10.133.0.61:15021: read: connection reset by peer"
Jul 07 23:45:44.316103 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:44.316067    2578 generic.go:356] "Generic (PLEG): container finished" podID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerID="5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323" exitCode=0
Jul 07 23:45:44.316615 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:44.316130    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" event={"ID":"1bf2ee95-a138-46c5-aacd-3205079b20ae","Type":"ContainerDied","Data":"5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323"}
Jul 07 23:45:44.316695 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:44.316677    2578 scope.go:117] "RemoveContainer" containerID="5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323"
Jul 07 23:45:44.317738 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:44.317710    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:44.317819 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:44.317785    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:44.317859 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:44.317818    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:45:45.323402 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:45.323365    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" event={"ID":"1bf2ee95-a138-46c5-aacd-3205079b20ae","Type":"ContainerStarted","Data":"90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845"}
Jul 07 23:45:45.535086 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:45.535041    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:45.535290 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:45.535265    2578 patch_prober.go:28] interesting pod/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 07 23:45:45.535337 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:45.535316    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 07 23:45:46.534542 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:46.534499    2578 patch_prober.go:28] interesting pod/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 07 23:45:46.534971 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:46.534591    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 07 23:45:47.535261 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:47.535227    2578 patch_prober.go:28] interesting pod/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 07 23:45:47.535672 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:47.535285    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 07 23:45:48.535234 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:48.535195    2578 patch_prober.go:28] interesting pod/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 07 23:45:48.535431 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:48.535252    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 07 23:45:49.538161 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:49.538129    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:49.538660 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:49.538486    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:45:49.539174 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:45:49.539156    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:46:00.025312 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:00.025267    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"]
Jul 07 23:46:00.025844 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:00.025525    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd" podUID="57f021cf-b2d0-4848-b14b-e99520e18d81" containerName="cleanup" containerID="cri-o://083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf" gracePeriod=30
Jul 07 23:46:30.234394 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.234361    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"
Jul 07 23:46:30.336351 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.336248    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfkq7\" (UniqueName: \"kubernetes.io/projected/57f021cf-b2d0-4848-b14b-e99520e18d81-kube-api-access-bfkq7\") pod \"57f021cf-b2d0-4848-b14b-e99520e18d81\" (UID: \"57f021cf-b2d0-4848-b14b-e99520e18d81\") "
Jul 07 23:46:30.338473 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.338439    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f021cf-b2d0-4848-b14b-e99520e18d81-kube-api-access-bfkq7" (OuterVolumeSpecName: "kube-api-access-bfkq7") pod "57f021cf-b2d0-4848-b14b-e99520e18d81" (UID: "57f021cf-b2d0-4848-b14b-e99520e18d81"). InnerVolumeSpecName "kube-api-access-bfkq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:46:30.437988 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.437940    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfkq7\" (UniqueName: \"kubernetes.io/projected/57f021cf-b2d0-4848-b14b-e99520e18d81-kube-api-access-bfkq7\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:46:30.565077 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.565038    2578 generic.go:356] "Generic (PLEG): container finished" podID="57f021cf-b2d0-4848-b14b-e99520e18d81" containerID="083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf" exitCode=137
Jul 07 23:46:30.565249 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.565086    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"
Jul 07 23:46:30.565249 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.565119    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd" event={"ID":"57f021cf-b2d0-4848-b14b-e99520e18d81","Type":"ContainerDied","Data":"083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf"}
Jul 07 23:46:30.565249 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.565192    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd" event={"ID":"57f021cf-b2d0-4848-b14b-e99520e18d81","Type":"ContainerDied","Data":"5d21139df56dcc6afe82684acbc6c8690f8cc6ff45929f44900395d681bb1ed3"}
Jul 07 23:46:30.565249 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.565223    2578 scope.go:117] "RemoveContainer" containerID="083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf"
Jul 07 23:46:30.577754 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.577461    2578 scope.go:117] "RemoveContainer" containerID="083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf"
Jul 07 23:46:30.578229 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:46:30.578203    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf\": container with ID starting with 083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf not found: ID does not exist" containerID="083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf"
Jul 07 23:46:30.578325 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.578241    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf"} err="failed to get container status \"083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf\": rpc error: code = NotFound desc = could not find container \"083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf\": container with ID starting with 083ceb58db6f6339d223e62f320278ccfeb2145d31f64d6072fd132356602acf not found: ID does not exist"
Jul 07 23:46:30.587789 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.587716    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"]
Jul 07 23:46:30.591995 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:30.591967    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724465-qxpwd"]
Jul 07 23:46:31.963383 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:31.963341    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57f021cf-b2d0-4848-b14b-e99520e18d81" path="/var/lib/kubelet/pods/57f021cf-b2d0-4848-b14b-e99520e18d81/volumes"
Jul 07 23:46:39.177064 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.177000    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"]
Jul 07 23:46:39.178087 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.178029    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws" podUID="c09431d5-b557-4883-9acd-69028260b1f7" containerName="maas-api" containerID="cri-o://b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd" gracePeriod=30
Jul 07 23:46:39.186715 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.186675    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws" podUID="c09431d5-b557-4883-9acd-69028260b1f7" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.60:8443/health\": dial tcp 10.133.0.60:8443: connect: connection refused"
Jul 07 23:46:39.436783 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.436702    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:46:39.526724 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.526688    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c09431d5-b557-4883-9acd-69028260b1f7-maas-api-tls\") pod \"c09431d5-b557-4883-9acd-69028260b1f7\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") "
Jul 07 23:46:39.526724 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.526731    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c09431d5-b557-4883-9acd-69028260b1f7-openshift-service-ca\") pod \"c09431d5-b557-4883-9acd-69028260b1f7\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") "
Jul 07 23:46:39.526947 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.526767    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmtcb\" (UniqueName: \"kubernetes.io/projected/c09431d5-b557-4883-9acd-69028260b1f7-kube-api-access-zmtcb\") pod \"c09431d5-b557-4883-9acd-69028260b1f7\" (UID: \"c09431d5-b557-4883-9acd-69028260b1f7\") "
Jul 07 23:46:39.527099 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.527075    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c09431d5-b557-4883-9acd-69028260b1f7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c09431d5-b557-4883-9acd-69028260b1f7" (UID: "c09431d5-b557-4883-9acd-69028260b1f7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:46:39.528901 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.528874    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c09431d5-b557-4883-9acd-69028260b1f7-kube-api-access-zmtcb" (OuterVolumeSpecName: "kube-api-access-zmtcb") pod "c09431d5-b557-4883-9acd-69028260b1f7" (UID: "c09431d5-b557-4883-9acd-69028260b1f7"). InnerVolumeSpecName "kube-api-access-zmtcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:46:39.528901 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.528887    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c09431d5-b557-4883-9acd-69028260b1f7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c09431d5-b557-4883-9acd-69028260b1f7" (UID: "c09431d5-b557-4883-9acd-69028260b1f7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:46:39.612751 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.612715    2578 generic.go:356] "Generic (PLEG): container finished" podID="c09431d5-b557-4883-9acd-69028260b1f7" containerID="b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd" exitCode=0
Jul 07 23:46:39.612919 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.612764    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"
Jul 07 23:46:39.612919 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.612801    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws" event={"ID":"c09431d5-b557-4883-9acd-69028260b1f7","Type":"ContainerDied","Data":"b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd"}
Jul 07 23:46:39.612919 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.612852    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws" event={"ID":"c09431d5-b557-4883-9acd-69028260b1f7","Type":"ContainerDied","Data":"2a917bb4990212ad50de331ce29d01cbe21f946795d26d00c7e9f011fe83bafa"}
Jul 07 23:46:39.612919 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.612883    2578 scope.go:117] "RemoveContainer" containerID="b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd"
Jul 07 23:46:39.622511 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.622483    2578 scope.go:117] "RemoveContainer" containerID="b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd"
Jul 07 23:46:39.622890 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:46:39.622869    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd\": container with ID starting with b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd not found: ID does not exist" containerID="b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd"
Jul 07 23:46:39.622998 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.622900    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd"} err="failed to get container status \"b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd\": rpc error: code = NotFound desc = could not find container \"b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd\": container with ID starting with b0dec7fd8b8662690eb08c552420c8654514b0b96e55d2d7560e9ff01057ebbd not found: ID does not exist"
Jul 07 23:46:39.627402 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.627382    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c09431d5-b557-4883-9acd-69028260b1f7-maas-api-tls\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:46:39.627402 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.627403    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c09431d5-b557-4883-9acd-69028260b1f7-openshift-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:46:39.627535 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.627415    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmtcb\" (UniqueName: \"kubernetes.io/projected/c09431d5-b557-4883-9acd-69028260b1f7-kube-api-access-zmtcb\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:46:39.635819 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.635792    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"]
Jul 07 23:46:39.641600 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.641574    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04f579-8db5d6bf9-csrws"]
Jul 07 23:46:39.963827 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:46:39.963796    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c09431d5-b557-4883-9acd-69028260b1f7" path="/var/lib/kubelet/pods/c09431d5-b557-4883-9acd-69028260b1f7/volumes"
Jul 07 23:47:20.902590 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:20.902532    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"]
Jul 07 23:47:20.903150 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:20.902823    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy" containerID="cri-o://f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4" gracePeriod=30
Jul 07 23:47:22.103977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.103940    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"]
Jul 07 23:47:22.104458 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.104156    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558" podUID="2903a1e8-9fe4-41d0-8c16-83515e0c64bd" containerName="maas-api" containerID="cri-o://1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347" gracePeriod=30
Jul 07 23:47:22.236942 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.236901    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558" podUID="2903a1e8-9fe4-41d0-8c16-83515e0c64bd" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.62:8443/health\": dial tcp 10.133.0.62:8443: connect: connection refused"
Jul 07 23:47:22.361013 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.360945    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:47:22.427807 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.427765    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-maas-api-tls\") pod \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") "
Jul 07 23:47:22.427984 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.427840    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-openshift-service-ca\") pod \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") "
Jul 07 23:47:22.427984 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.427884    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfqbk\" (UniqueName: \"kubernetes.io/projected/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-kube-api-access-pfqbk\") pod \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\" (UID: \"2903a1e8-9fe4-41d0-8c16-83515e0c64bd\") "
Jul 07 23:47:22.428291 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.428259    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2903a1e8-9fe4-41d0-8c16-83515e0c64bd" (UID: "2903a1e8-9fe4-41d0-8c16-83515e0c64bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:47:22.430040 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.430014    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-kube-api-access-pfqbk" (OuterVolumeSpecName: "kube-api-access-pfqbk") pod "2903a1e8-9fe4-41d0-8c16-83515e0c64bd" (UID: "2903a1e8-9fe4-41d0-8c16-83515e0c64bd"). InnerVolumeSpecName "kube-api-access-pfqbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:47:22.430131 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.430041    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2903a1e8-9fe4-41d0-8c16-83515e0c64bd" (UID: "2903a1e8-9fe4-41d0-8c16-83515e0c64bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:47:22.528914 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.528873    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-openshift-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:22.528914 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.528909    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfqbk\" (UniqueName: \"kubernetes.io/projected/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-kube-api-access-pfqbk\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:22.528914 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.528919    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2903a1e8-9fe4-41d0-8c16-83515e0c64bd-maas-api-tls\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:22.836513 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.836477    2578 generic.go:356] "Generic (PLEG): container finished" podID="2903a1e8-9fe4-41d0-8c16-83515e0c64bd" containerID="1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347" exitCode=0
Jul 07 23:47:22.836751 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.836530    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"
Jul 07 23:47:22.836751 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.836574    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558" event={"ID":"2903a1e8-9fe4-41d0-8c16-83515e0c64bd","Type":"ContainerDied","Data":"1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347"}
Jul 07 23:47:22.836751 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.836666    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558" event={"ID":"2903a1e8-9fe4-41d0-8c16-83515e0c64bd","Type":"ContainerDied","Data":"d9af356affe433f52d7ec89874fd390f1d38433b794140a878fecbf2d35f9c7f"}
Jul 07 23:47:22.836751 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.836696    2578 scope.go:117] "RemoveContainer" containerID="1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347"
Jul 07 23:47:22.845851 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.845831    2578 scope.go:117] "RemoveContainer" containerID="1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347"
Jul 07 23:47:22.846113 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:47:22.846094    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347\": container with ID starting with 1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347 not found: ID does not exist" containerID="1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347"
Jul 07 23:47:22.846177 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.846122    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347"} err="failed to get container status \"1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347\": rpc error: code = NotFound desc = could not find container \"1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347\": container with ID starting with 1c066cdad64015d8db7c138ed74018e45e514b9cebf144cc5cb25b58db7d8347 not found: ID does not exist"
Jul 07 23:47:22.860971 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.860937    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"]
Jul 07 23:47:22.865083 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:22.865055    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c324fe-679cc7d55d-xx558"]
Jul 07 23:47:23.963804 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:23.963774    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2903a1e8-9fe4-41d0-8c16-83515e0c64bd" path="/var/lib/kubelet/pods/2903a1e8-9fe4-41d0-8c16-83515e0c64bd/volumes"
Jul 07 23:47:26.167624 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.167545    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:47:26.255337 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255296    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-envoy\") pod \"2299d28c-4d51-4adf-8878-a801c16f43a7\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") "
Jul 07 23:47:26.255337 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255339    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2299d28c-4d51-4adf-8878-a801c16f43a7-istiod-ca-cert\") pod \"2299d28c-4d51-4adf-8878-a801c16f43a7\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") "
Jul 07 23:47:26.255637 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255386    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-credential-socket\") pod \"2299d28c-4d51-4adf-8878-a801c16f43a7\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") "
Jul 07 23:47:26.255637 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255411    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-certs\") pod \"2299d28c-4d51-4adf-8878-a801c16f43a7\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") "
Jul 07 23:47:26.255637 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255434    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-data\") pod \"2299d28c-4d51-4adf-8878-a801c16f43a7\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") "
Jul 07 23:47:26.255637 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255455    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-socket\") pod \"2299d28c-4d51-4adf-8878-a801c16f43a7\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") "
Jul 07 23:47:26.255637 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255516    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-token\") pod \"2299d28c-4d51-4adf-8878-a801c16f43a7\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") "
Jul 07 23:47:26.255637 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255540    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-podinfo\") pod \"2299d28c-4d51-4adf-8878-a801c16f43a7\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") "
Jul 07 23:47:26.255637 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255610    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf4jv\" (UniqueName: \"kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-kube-api-access-xf4jv\") pod \"2299d28c-4d51-4adf-8878-a801c16f43a7\" (UID: \"2299d28c-4d51-4adf-8878-a801c16f43a7\") "
Jul 07 23:47:26.256007 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255777    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2299d28c-4d51-4adf-8878-a801c16f43a7" (UID: "2299d28c-4d51-4adf-8878-a801c16f43a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:47:26.256007 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255796    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2299d28c-4d51-4adf-8878-a801c16f43a7" (UID: "2299d28c-4d51-4adf-8878-a801c16f43a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:47:26.256007 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255885    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2299d28c-4d51-4adf-8878-a801c16f43a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2299d28c-4d51-4adf-8878-a801c16f43a7" (UID: "2299d28c-4d51-4adf-8878-a801c16f43a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:47:26.256007 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255967    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2299d28c-4d51-4adf-8878-a801c16f43a7" (UID: "2299d28c-4d51-4adf-8878-a801c16f43a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:47:26.256007 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.255981    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-credential-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:26.256007 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.256001    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:26.256336 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.256016    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2299d28c-4d51-4adf-8878-a801c16f43a7-istiod-ca-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:26.256497 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.256470    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "2299d28c-4d51-4adf-8878-a801c16f43a7" (UID: "2299d28c-4d51-4adf-8878-a801c16f43a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:47:26.257722 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.257697    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2299d28c-4d51-4adf-8878-a801c16f43a7" (UID: "2299d28c-4d51-4adf-8878-a801c16f43a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:47:26.257860 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.257835    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "2299d28c-4d51-4adf-8878-a801c16f43a7" (UID: "2299d28c-4d51-4adf-8878-a801c16f43a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:47:26.257920 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.257862    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-kube-api-access-xf4jv" (OuterVolumeSpecName: "kube-api-access-xf4jv") pod "2299d28c-4d51-4adf-8878-a801c16f43a7" (UID: "2299d28c-4d51-4adf-8878-a801c16f43a7"). InnerVolumeSpecName "kube-api-access-xf4jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:47:26.258071 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.258051    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2299d28c-4d51-4adf-8878-a801c16f43a7" (UID: "2299d28c-4d51-4adf-8878-a801c16f43a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 23:47:26.356514 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.356482    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-data\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:26.356514 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.356515    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-workload-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:26.356765 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.356531    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:26.356765 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.356544    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-podinfo\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:26.356765 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.356572    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf4jv\" (UniqueName: \"kubernetes.io/projected/2299d28c-4d51-4adf-8878-a801c16f43a7-kube-api-access-xf4jv\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:26.356765 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.356588    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2299d28c-4d51-4adf-8878-a801c16f43a7-istio-envoy\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:26.859271 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.859233    2578 generic.go:356] "Generic (PLEG): container finished" podID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerID="f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4" exitCode=0
Jul 07 23:47:26.859472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.859284    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"
Jul 07 23:47:26.859472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.859323    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" event={"ID":"2299d28c-4d51-4adf-8878-a801c16f43a7","Type":"ContainerDied","Data":"f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4"}
Jul 07 23:47:26.859472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.859383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh" event={"ID":"2299d28c-4d51-4adf-8878-a801c16f43a7","Type":"ContainerDied","Data":"2ae02fae828360a866a0615a061a3b8a24f7a1a4294f421227e1e17e41813fbc"}
Jul 07 23:47:26.859472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.859408    2578 scope.go:117] "RemoveContainer" containerID="f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4"
Jul 07 23:47:26.869535 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.869515    2578 scope.go:117] "RemoveContainer" containerID="bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6"
Jul 07 23:47:26.878238 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.878216    2578 scope.go:117] "RemoveContainer" containerID="f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4"
Jul 07 23:47:26.878534 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:47:26.878514    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4\": container with ID starting with f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4 not found: ID does not exist" containerID="f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4"
Jul 07 23:47:26.878727 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.878548    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4"} err="failed to get container status \"f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4\": rpc error: code = NotFound desc = could not find container \"f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4\": container with ID starting with f5578717a44fc3ef45b1000b6c42d8d70797a0c4bbbd59741fb1e93b1aa16dd4 not found: ID does not exist"
Jul 07 23:47:26.878727 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.878601    2578 scope.go:117] "RemoveContainer" containerID="bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6"
Jul 07 23:47:26.878868 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:47:26.878852    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6\": container with ID starting with bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6 not found: ID does not exist" containerID="bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6"
Jul 07 23:47:26.878921 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.878875    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6"} err="failed to get container status \"bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6\": rpc error: code = NotFound desc = could not find container \"bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6\": container with ID starting with bf043f3054332aef2d1368d3195b2321fa85c5b28ab3a410ed2789e323ef33f6 not found: ID does not exist"
Jul 07 23:47:26.881875 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.881846    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"]
Jul 07 23:47:26.888410 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:26.888385    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-04f579-openshift-default-6f89f6fd99-2tgzh"]
Jul 07 23:47:27.963661 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:27.963628    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" path="/var/lib/kubelet/pods/2299d28c-4d51-4adf-8878-a801c16f43a7/volumes"
Jul 07 23:47:28.440758 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:28.440720    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"]
Jul 07 23:47:28.440963 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:28.440921    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy" containerID="cri-o://90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845" gracePeriod=30
Jul 07 23:47:29.331613 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.331578    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"]
Jul 07 23:47:29.332147 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332132    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2903a1e8-9fe4-41d0-8c16-83515e0c64bd" containerName="maas-api"
Jul 07 23:47:29.332197 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332151    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2903a1e8-9fe4-41d0-8c16-83515e0c64bd" containerName="maas-api"
Jul 07 23:47:29.332197 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332183    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy"
Jul 07 23:47:29.332197 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332191    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy"
Jul 07 23:47:29.332306 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332199    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c09431d5-b557-4883-9acd-69028260b1f7" containerName="maas-api"
Jul 07 23:47:29.332306 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332204    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c09431d5-b557-4883-9acd-69028260b1f7" containerName="maas-api"
Jul 07 23:47:29.332306 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332213    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57f021cf-b2d0-4848-b14b-e99520e18d81" containerName="cleanup"
Jul 07 23:47:29.332306 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332218    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="57f021cf-b2d0-4848-b14b-e99520e18d81" containerName="cleanup"
Jul 07 23:47:29.332306 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332228    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy"
Jul 07 23:47:29.332306 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332233    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy"
Jul 07 23:47:29.332306 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332295    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="57f021cf-b2d0-4848-b14b-e99520e18d81" containerName="cleanup"
Jul 07 23:47:29.332306 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332303    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy"
Jul 07 23:47:29.332306 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332309    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2299d28c-4d51-4adf-8878-a801c16f43a7" containerName="istio-proxy"
Jul 07 23:47:29.332600 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332315    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c09431d5-b557-4883-9acd-69028260b1f7" containerName="maas-api"
Jul 07 23:47:29.332600 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.332323    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2903a1e8-9fe4-41d0-8c16-83515e0c64bd" containerName="maas-api"
Jul 07 23:47:29.337479 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.337460    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.340484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.340462    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-6e6351-openshift-default-dockercfg-rq4gp\""
Jul 07 23:47:29.355262 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.355231    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"]
Jul 07 23:47:29.379391 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.379333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvc96\" (UniqueName: \"kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-kube-api-access-vvc96\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.379391 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.379399    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istiod-ca-cert\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.379668 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.379450    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-credential-socket\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.379668 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.379520    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-podinfo\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.379668 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.379587    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-token\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.379668 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.379646    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-certs\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.379842 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.379683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-envoy\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.379842 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.379734    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-data\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.379842 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.379766    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-socket\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.481307 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.481259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-credential-socket\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.481509 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.481332    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-podinfo\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.481509 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.481372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-token\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.481509 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.481413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-certs\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.481818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.481540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-envoy\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.481818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.481619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-data\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.481818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.481661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-socket\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.481818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.481705    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvc96\" (UniqueName: \"kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-kube-api-access-vvc96\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.481818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.481736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istiod-ca-cert\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.481818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.481768    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-credential-socket\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.481818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.481775    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-certs\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.482186 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.482017    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-data\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.482186 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.482139    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-socket\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.482436 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.482417    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istiod-ca-cert\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.483983 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.483962    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-podinfo\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.484112 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.484094    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-envoy\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.490219 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.490190    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-token\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.490478 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.490457    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvc96\" (UniqueName: \"kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-kube-api-access-vvc96\") pod \"e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") " pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.647728 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.647690    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:29.798191 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.798104    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"]
Jul 07 23:47:29.800711 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:47:29.800686    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fd8ce9d_85ff_4330_aae4_ce92b2ab6208.slice/crio-5932f96fbeab0762f74f7b33cae6f6b042b3af1c59be064eefb57b6429a5708b WatchSource:0}: Error finding container 5932f96fbeab0762f74f7b33cae6f6b042b3af1c59be064eefb57b6429a5708b: Status 404 returned error can't find the container with id 5932f96fbeab0762f74f7b33cae6f6b042b3af1c59be064eefb57b6429a5708b
Jul 07 23:47:29.803413 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.803377    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:29.803502 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.803465    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:29.803553 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.803504    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:29.878105 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.878074    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" event={"ID":"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208","Type":"ContainerStarted","Data":"0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079"}
Jul 07 23:47:29.878248 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.878137    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" event={"ID":"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208","Type":"ContainerStarted","Data":"5932f96fbeab0762f74f7b33cae6f6b042b3af1c59be064eefb57b6429a5708b"}
Jul 07 23:47:29.899492 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:29.899386    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" podStartSLOduration=0.89936836 podStartE2EDuration="899.36836ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:47:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:47:29.89882291 +0000 UTC m=+2034.484392273" watchObservedRunningTime="2026-07-07 23:47:29.89936836 +0000 UTC m=+2034.484937721"
Jul 07 23:47:30.648250 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:30.648210    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:30.653309 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:30.653275    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:30.883421 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:30.883395    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:30.884653 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:30.884636    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:31.954963 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:31.954919    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"]
Jul 07 23:47:31.958915 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:31.958888    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:31.961618 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:31.961594    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6e6351-serving-cert\""
Jul 07 23:47:31.970328 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:31.970297    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"]
Jul 07 23:47:32.005891 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:32.005857    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a28e237-d8ad-40ad-a17c-807958349f14-openshift-service-ca\") pod \"maas-api-e2e-api-a-6e6351-d89b47547-pgz5p\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:32.006070 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:32.005919    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a28e237-d8ad-40ad-a17c-807958349f14-maas-api-tls\") pod \"maas-api-e2e-api-a-6e6351-d89b47547-pgz5p\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:32.006070 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:32.005965    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqsdp\" (UniqueName: \"kubernetes.io/projected/4a28e237-d8ad-40ad-a17c-807958349f14-kube-api-access-rqsdp\") pod \"maas-api-e2e-api-a-6e6351-d89b47547-pgz5p\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:32.106712 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:32.106668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a28e237-d8ad-40ad-a17c-807958349f14-maas-api-tls\") pod \"maas-api-e2e-api-a-6e6351-d89b47547-pgz5p\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:32.106712 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:32.106733    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqsdp\" (UniqueName: \"kubernetes.io/projected/4a28e237-d8ad-40ad-a17c-807958349f14-kube-api-access-rqsdp\") pod \"maas-api-e2e-api-a-6e6351-d89b47547-pgz5p\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:32.107047 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:32.106842    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a28e237-d8ad-40ad-a17c-807958349f14-openshift-service-ca\") pod \"maas-api-e2e-api-a-6e6351-d89b47547-pgz5p\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:32.107763 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:47:32.107553    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-serving-cert: secret "maas-api-e2e-api-a-6e6351-serving-cert" not found
Jul 07 23:47:32.107763 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:47:32.107656    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a28e237-d8ad-40ad-a17c-807958349f14-maas-api-tls podName:4a28e237-d8ad-40ad-a17c-807958349f14 nodeName:}" failed. No retries permitted until 2026-07-07 23:47:32.607632018 +0000 UTC m=+2037.193201362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4a28e237-d8ad-40ad-a17c-807958349f14-maas-api-tls") pod "maas-api-e2e-api-a-6e6351-d89b47547-pgz5p" (UID: "4a28e237-d8ad-40ad-a17c-807958349f14") : secret "maas-api-e2e-api-a-6e6351-serving-cert" not found
Jul 07 23:47:32.107763 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:32.107657    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a28e237-d8ad-40ad-a17c-807958349f14-openshift-service-ca\") pod \"maas-api-e2e-api-a-6e6351-d89b47547-pgz5p\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:32.118109 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:32.118078    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqsdp\" (UniqueName: \"kubernetes.io/projected/4a28e237-d8ad-40ad-a17c-807958349f14-kube-api-access-rqsdp\") pod \"maas-api-e2e-api-a-6e6351-d89b47547-pgz5p\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:32.612498 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:32.612457    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a28e237-d8ad-40ad-a17c-807958349f14-maas-api-tls\") pod \"maas-api-e2e-api-a-6e6351-d89b47547-pgz5p\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:32.615034 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:32.615011    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a28e237-d8ad-40ad-a17c-807958349f14-maas-api-tls\") pod \"maas-api-e2e-api-a-6e6351-d89b47547-pgz5p\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:32.872003 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:32.871912    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:33.019219 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.019195    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"]
Jul 07 23:47:33.021507 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:47:33.021482    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a28e237_d8ad_40ad_a17c_807958349f14.slice/crio-fb5e1062fdbea63622f7d6e2cd7a68d69e69e98e381c757527ceae1a73679f15 WatchSource:0}: Error finding container fb5e1062fdbea63622f7d6e2cd7a68d69e69e98e381c757527ceae1a73679f15: Status 404 returned error can't find the container with id fb5e1062fdbea63622f7d6e2cd7a68d69e69e98e381c757527ceae1a73679f15
Jul 07 23:47:33.703823 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.703754    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:47:33.722909 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.722870    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf2ee95-a138-46c5-aacd-3205079b20ae-istiod-ca-cert\") pod \"1bf2ee95-a138-46c5-aacd-3205079b20ae\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") "
Jul 07 23:47:33.723117 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.722947    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-envoy\") pod \"1bf2ee95-a138-46c5-aacd-3205079b20ae\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") "
Jul 07 23:47:33.723117 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.722966    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-socket\") pod \"1bf2ee95-a138-46c5-aacd-3205079b20ae\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") "
Jul 07 23:47:33.723117 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.722984    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-credential-socket\") pod \"1bf2ee95-a138-46c5-aacd-3205079b20ae\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") "
Jul 07 23:47:33.723117 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723003    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-certs\") pod \"1bf2ee95-a138-46c5-aacd-3205079b20ae\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") "
Jul 07 23:47:33.723117 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723022    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-data\") pod \"1bf2ee95-a138-46c5-aacd-3205079b20ae\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") "
Jul 07 23:47:33.723117 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723071    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxmgt\" (UniqueName: \"kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-kube-api-access-lxmgt\") pod \"1bf2ee95-a138-46c5-aacd-3205079b20ae\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") "
Jul 07 23:47:33.723117 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723107    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-token\") pod \"1bf2ee95-a138-46c5-aacd-3205079b20ae\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") "
Jul 07 23:47:33.723472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723203    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-podinfo\") pod \"1bf2ee95-a138-46c5-aacd-3205079b20ae\" (UID: \"1bf2ee95-a138-46c5-aacd-3205079b20ae\") "
Jul 07 23:47:33.723472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723287    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1bf2ee95-a138-46c5-aacd-3205079b20ae" (UID: "1bf2ee95-a138-46c5-aacd-3205079b20ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:47:33.723472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723325    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1bf2ee95-a138-46c5-aacd-3205079b20ae" (UID: "1bf2ee95-a138-46c5-aacd-3205079b20ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:47:33.723472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723332    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1bf2ee95-a138-46c5-aacd-3205079b20ae" (UID: "1bf2ee95-a138-46c5-aacd-3205079b20ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:47:33.723472 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723353    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf2ee95-a138-46c5-aacd-3205079b20ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1bf2ee95-a138-46c5-aacd-3205079b20ae" (UID: "1bf2ee95-a138-46c5-aacd-3205079b20ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:47:33.723718 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723546    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf2ee95-a138-46c5-aacd-3205079b20ae-istiod-ca-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:33.723718 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723598    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:33.723718 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723613    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-credential-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:33.723718 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.723628    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-workload-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:33.724085 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.724059    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "1bf2ee95-a138-46c5-aacd-3205079b20ae" (UID: "1bf2ee95-a138-46c5-aacd-3205079b20ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:47:33.725454 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.725424    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-kube-api-access-lxmgt" (OuterVolumeSpecName: "kube-api-access-lxmgt") pod "1bf2ee95-a138-46c5-aacd-3205079b20ae" (UID: "1bf2ee95-a138-46c5-aacd-3205079b20ae"). InnerVolumeSpecName "kube-api-access-lxmgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:47:33.725454 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.725440    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1bf2ee95-a138-46c5-aacd-3205079b20ae" (UID: "1bf2ee95-a138-46c5-aacd-3205079b20ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 23:47:33.725666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.725503    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "1bf2ee95-a138-46c5-aacd-3205079b20ae" (UID: "1bf2ee95-a138-46c5-aacd-3205079b20ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:47:33.725666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.725522    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1bf2ee95-a138-46c5-aacd-3205079b20ae" (UID: "1bf2ee95-a138-46c5-aacd-3205079b20ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:47:33.825103 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.825059    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-envoy\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:33.825103 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.825094    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-data\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:33.825103 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.825109    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxmgt\" (UniqueName: \"kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-kube-api-access-lxmgt\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:33.825363 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.825121    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:33.825363 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.825134    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf2ee95-a138-46c5-aacd-3205079b20ae-istio-podinfo\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:33.901620 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.901549    2578 generic.go:356] "Generic (PLEG): container finished" podID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerID="90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845" exitCode=0
Jul 07 23:47:33.901620 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.901602    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" event={"ID":"1bf2ee95-a138-46c5-aacd-3205079b20ae","Type":"ContainerDied","Data":"90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845"}
Jul 07 23:47:33.901845 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.901636    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"
Jul 07 23:47:33.901845 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.901663    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422" event={"ID":"1bf2ee95-a138-46c5-aacd-3205079b20ae","Type":"ContainerDied","Data":"ef3675c7780a48950c5ada5283e462ccf53dbd9b40632a083760c54b159e9bc3"}
Jul 07 23:47:33.901845 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.901687    2578 scope.go:117] "RemoveContainer" containerID="90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845"
Jul 07 23:47:33.903259 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.903232    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p" event={"ID":"4a28e237-d8ad-40ad-a17c-807958349f14","Type":"ContainerStarted","Data":"46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b"}
Jul 07 23:47:33.903365 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.903336    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p" event={"ID":"4a28e237-d8ad-40ad-a17c-807958349f14","Type":"ContainerStarted","Data":"fb5e1062fdbea63622f7d6e2cd7a68d69e69e98e381c757527ceae1a73679f15"}
Jul 07 23:47:33.903424 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.903389    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:33.912434 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.912416    2578 scope.go:117] "RemoveContainer" containerID="5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323"
Jul 07 23:47:33.923276 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.923254    2578 scope.go:117] "RemoveContainer" containerID="90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845"
Jul 07 23:47:33.923647 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:47:33.923594    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845\": container with ID starting with 90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845 not found: ID does not exist" containerID="90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845"
Jul 07 23:47:33.923647 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.923630    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845"} err="failed to get container status \"90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845\": rpc error: code = NotFound desc = could not find container \"90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845\": container with ID starting with 90e63edc659921b89ac1490112e97cfd7eeab9d7fd3b94c98c936dcab5e4f845 not found: ID does not exist"
Jul 07 23:47:33.923647 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.923648    2578 scope.go:117] "RemoveContainer" containerID="5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323"
Jul 07 23:47:33.923942 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:47:33.923923    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323\": container with ID starting with 5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323 not found: ID does not exist" containerID="5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323"
Jul 07 23:47:33.924002 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.923951    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323"} err="failed to get container status \"5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323\": rpc error: code = NotFound desc = could not find container \"5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323\": container with ID starting with 5319025a8992f2c4982a638de8a7d9bb8a216c5c72da566e44502e300964c323 not found: ID does not exist"
Jul 07 23:47:33.928832 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.928787    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p" podStartSLOduration=2.338203198 podStartE2EDuration="2.928773808s" totalImagesPullingTime="590.57061ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:47:33.927993277 +0000 UTC m=+2038.513562639" watchObservedRunningTime="2026-07-07 23:47:33.928773808 +0000 UTC m=+2038.514343169"
Jul 07 23:47:33.942815 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.942780    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"]
Jul 07 23:47:33.948260 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.948229    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c324fe-openshift-default-6c7d7ffd5-wd422"]
Jul 07 23:47:33.962910 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:33.962833    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" path="/var/lib/kubelet/pods/1bf2ee95-a138-46c5-aacd-3205079b20ae/volumes"
Jul 07 23:47:37.022225 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.022188    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"]
Jul 07 23:47:37.022832 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.022810    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy"
Jul 07 23:47:37.022921 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.022832    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy"
Jul 07 23:47:37.022921 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.022855    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy"
Jul 07 23:47:37.022921 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.022864    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy"
Jul 07 23:47:37.023139 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.022966    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy"
Jul 07 23:47:37.023139 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.022978    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bf2ee95-a138-46c5-aacd-3205079b20ae" containerName="istio-proxy"
Jul 07 23:47:37.027944 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.027924    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.034578 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.032156    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b9fe5e-openshift-default-dockercfg-pdldh\""
Jul 07 23:47:37.041635 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.041515    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"]
Jul 07 23:47:37.056069 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.056036    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-envoy\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.056234 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.056075    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-credential-socket\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.056234 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.056112    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef61c21-6b32-4677-84e8-67d32e135a16-istiod-ca-cert\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.056234 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.056128    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcxgk\" (UniqueName: \"kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-kube-api-access-wcxgk\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.056234 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.056147    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-certs\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.056234 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.056166    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-data\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.056234 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.056227    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-socket\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.056470 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.056257    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-token\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.056470 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.056308    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-podinfo\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.156774 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.156734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-credential-socket\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.156774 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.156779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef61c21-6b32-4677-84e8-67d32e135a16-istiod-ca-cert\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157033 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.156795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcxgk\" (UniqueName: \"kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-kube-api-access-wcxgk\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157033 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.156812    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-certs\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157033 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.156828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-data\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157033 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.156857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-socket\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157033 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.156879    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-token\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157033 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.156913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-podinfo\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157033 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.156978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-envoy\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157476 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.157329    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-socket\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157476 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.157425    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-credential-socket\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157476 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.157471    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-certs\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157659 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.157573    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-data\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.157659 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.157596    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef61c21-6b32-4677-84e8-67d32e135a16-istiod-ca-cert\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.159340 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.159319    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-envoy\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.159527 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.159509    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-podinfo\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.165345 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.165324    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-token\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.165451 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.165352    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcxgk\" (UniqueName: \"kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-kube-api-access-wcxgk\") pod \"e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") " pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.348131 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.348030    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:37.492501 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.492473    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"]
Jul 07 23:47:37.494212 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:47:37.494183    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef61c21_6b32_4677_84e8_67d32e135a16.slice/crio-1facab782388a6a60cf038235c077979718e61fc7e28b144333b2f8bd91545c9 WatchSource:0}: Error finding container 1facab782388a6a60cf038235c077979718e61fc7e28b144333b2f8bd91545c9: Status 404 returned error can't find the container with id 1facab782388a6a60cf038235c077979718e61fc7e28b144333b2f8bd91545c9
Jul 07 23:47:37.496423 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.496391    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:37.496508 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.496459    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:37.496508 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.496485    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:37.926213 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.926178    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" event={"ID":"1ef61c21-6b32-4677-84e8-67d32e135a16","Type":"ContainerStarted","Data":"13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251"}
Jul 07 23:47:37.926415 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.926234    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" event={"ID":"1ef61c21-6b32-4677-84e8-67d32e135a16","Type":"ContainerStarted","Data":"1facab782388a6a60cf038235c077979718e61fc7e28b144333b2f8bd91545c9"}
Jul 07 23:47:37.950727 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:37.950668    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" podStartSLOduration=0.950624267 podStartE2EDuration="950.624267ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:47:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:47:37.949014638 +0000 UTC m=+2042.534583999" watchObservedRunningTime="2026-07-07 23:47:37.950624267 +0000 UTC m=+2042.536193628"
Jul 07 23:47:38.349265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:38.349166    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:38.354253 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:38.354227    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:38.931857 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:38.931823    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:38.932749 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:38.932728    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:39.914938 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:39.914910    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:39.966267 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:39.966232    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"]
Jul 07 23:47:39.970528 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:39.970504    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:39.975440 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:39.975409    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b9fe5e-serving-cert\""
Jul 07 23:47:39.976294 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:39.976217    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"]
Jul 07 23:47:40.089245 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:40.089202    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-openshift-service-ca\") pod \"maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:40.089459 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:40.089304    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjvf2\" (UniqueName: \"kubernetes.io/projected/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-kube-api-access-wjvf2\") pod \"maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:40.089459 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:40.089348    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-maas-api-tls\") pod \"maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:40.190766 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:40.190669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-openshift-service-ca\") pod \"maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:40.190766 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:40.190741    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjvf2\" (UniqueName: \"kubernetes.io/projected/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-kube-api-access-wjvf2\") pod \"maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:40.190766 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:40.190767    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-maas-api-tls\") pod \"maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:40.190997 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:47:40.190872    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-serving-cert: secret "maas-api-e2e-api-b-b9fe5e-serving-cert" not found
Jul 07 23:47:40.190997 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:47:40.190936    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-maas-api-tls podName:115c9f0e-65ac-4b93-8b6d-e4571fc99c97 nodeName:}" failed. No retries permitted until 2026-07-07 23:47:40.690917502 +0000 UTC m=+2045.276486848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-maas-api-tls") pod "maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2" (UID: "115c9f0e-65ac-4b93-8b6d-e4571fc99c97") : secret "maas-api-e2e-api-b-b9fe5e-serving-cert" not found
Jul 07 23:47:40.191522 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:40.191500    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-openshift-service-ca\") pod \"maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:40.203097 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:40.203058    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjvf2\" (UniqueName: \"kubernetes.io/projected/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-kube-api-access-wjvf2\") pod \"maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:40.696395 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:40.696353    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-maas-api-tls\") pod \"maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:40.699153 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:40.699119    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-maas-api-tls\") pod \"maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:40.886396 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:40.886363    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:41.047244 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:41.047209    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"]
Jul 07 23:47:41.048730 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:47:41.048700    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115c9f0e_65ac_4b93_8b6d_e4571fc99c97.slice/crio-b5546898a1549c125a8463d84bb4b622094386249229f5a9137d9670152287d1 WatchSource:0}: Error finding container b5546898a1549c125a8463d84bb4b622094386249229f5a9137d9670152287d1: Status 404 returned error can't find the container with id b5546898a1549c125a8463d84bb4b622094386249229f5a9137d9670152287d1
Jul 07 23:47:41.952836 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:41.952797    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2" event={"ID":"115c9f0e-65ac-4b93-8b6d-e4571fc99c97","Type":"ContainerStarted","Data":"229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247"}
Jul 07 23:47:41.953019 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:41.952864    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2" event={"ID":"115c9f0e-65ac-4b93-8b6d-e4571fc99c97","Type":"ContainerStarted","Data":"b5546898a1549c125a8463d84bb4b622094386249229f5a9137d9670152287d1"}
Jul 07 23:47:41.953090 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:41.953023    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:41.971314 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:41.971264    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2" podStartSLOduration=2.661961235 podStartE2EDuration="2.971246311s" totalImagesPullingTime="309.285076ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:47:41.969602738 +0000 UTC m=+2046.555172094" watchObservedRunningTime="2026-07-07 23:47:41.971246311 +0000 UTC m=+2046.556815672"
Jul 07 23:47:46.925735 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:46.925690    2578 patch_prober.go:28] interesting pod/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 23:47:46.926136 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:46.925763    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 23:47:47.965143 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:47.965100    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:47:48.998407 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:48.998367    2578 generic.go:356] "Generic (PLEG): container finished" podID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerID="0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079" exitCode=0
Jul 07 23:47:48.999069 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:48.999042    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" event={"ID":"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208","Type":"ContainerDied","Data":"0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079"}
Jul 07 23:47:48.999689 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:48.999669    2578 scope.go:117] "RemoveContainer" containerID="0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079"
Jul 07 23:47:49.001896 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:49.001860    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:49.002102 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:49.002082    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:49.002243 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:49.002227    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:50.010651 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:50.010613    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" event={"ID":"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208","Type":"ContainerStarted","Data":"159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5"}
Jul 07 23:47:50.649026 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:50.648985    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:50.649214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:50.649092    2578 patch_prober.go:28] interesting pod/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 07 23:47:50.649214 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:50.649127    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 07 23:47:51.648667 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:51.648627    2578 patch_prober.go:28] interesting pod/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 07 23:47:51.649111 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:51.648695    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 07 23:47:52.649179 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:52.649135    2578 patch_prober.go:28] interesting pod/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 07 23:47:52.649674 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:52.649205    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 07 23:47:53.278185 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.278145    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"]
Jul 07 23:47:53.278717 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.278683    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p" podUID="4a28e237-d8ad-40ad-a17c-807958349f14" containerName="maas-api" containerID="cri-o://46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b" gracePeriod=30
Jul 07 23:47:53.608181 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.607802    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:53.649196 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.649151    2578 patch_prober.go:28] interesting pod/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 07 23:47:53.649700 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.649229    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 07 23:47:53.736749 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.736713    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a28e237-d8ad-40ad-a17c-807958349f14-maas-api-tls\") pod \"4a28e237-d8ad-40ad-a17c-807958349f14\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") "
Jul 07 23:47:53.736927 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.736778    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqsdp\" (UniqueName: \"kubernetes.io/projected/4a28e237-d8ad-40ad-a17c-807958349f14-kube-api-access-rqsdp\") pod \"4a28e237-d8ad-40ad-a17c-807958349f14\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") "
Jul 07 23:47:53.736927 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.736809    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a28e237-d8ad-40ad-a17c-807958349f14-openshift-service-ca\") pod \"4a28e237-d8ad-40ad-a17c-807958349f14\" (UID: \"4a28e237-d8ad-40ad-a17c-807958349f14\") "
Jul 07 23:47:53.737404 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.737370    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a28e237-d8ad-40ad-a17c-807958349f14-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4a28e237-d8ad-40ad-a17c-807958349f14" (UID: "4a28e237-d8ad-40ad-a17c-807958349f14"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:47:53.739030 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.739000    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a28e237-d8ad-40ad-a17c-807958349f14-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4a28e237-d8ad-40ad-a17c-807958349f14" (UID: "4a28e237-d8ad-40ad-a17c-807958349f14"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:47:53.739955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.739929    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a28e237-d8ad-40ad-a17c-807958349f14-kube-api-access-rqsdp" (OuterVolumeSpecName: "kube-api-access-rqsdp") pod "4a28e237-d8ad-40ad-a17c-807958349f14" (UID: "4a28e237-d8ad-40ad-a17c-807958349f14"). InnerVolumeSpecName "kube-api-access-rqsdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:47:53.838244 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.838145    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a28e237-d8ad-40ad-a17c-807958349f14-maas-api-tls\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:53.838244 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.838180    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqsdp\" (UniqueName: \"kubernetes.io/projected/4a28e237-d8ad-40ad-a17c-807958349f14-kube-api-access-rqsdp\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:53.838244 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:53.838191    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a28e237-d8ad-40ad-a17c-807958349f14-openshift-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:47:54.038720 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.038687    2578 generic.go:356] "Generic (PLEG): container finished" podID="4a28e237-d8ad-40ad-a17c-807958349f14" containerID="46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b" exitCode=0
Jul 07 23:47:54.038922 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.038764    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"
Jul 07 23:47:54.038922 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.038786    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p" event={"ID":"4a28e237-d8ad-40ad-a17c-807958349f14","Type":"ContainerDied","Data":"46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b"}
Jul 07 23:47:54.038922 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.038833    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p" event={"ID":"4a28e237-d8ad-40ad-a17c-807958349f14","Type":"ContainerDied","Data":"fb5e1062fdbea63622f7d6e2cd7a68d69e69e98e381c757527ceae1a73679f15"}
Jul 07 23:47:54.038922 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.038853    2578 scope.go:117] "RemoveContainer" containerID="46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b"
Jul 07 23:47:54.049947 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.049923    2578 scope.go:117] "RemoveContainer" containerID="46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b"
Jul 07 23:47:54.050329 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:47:54.050297    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b\": container with ID starting with 46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b not found: ID does not exist" containerID="46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b"
Jul 07 23:47:54.050426 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.050337    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b"} err="failed to get container status \"46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b\": rpc error: code = NotFound desc = could not find container \"46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b\": container with ID starting with 46327a2114527dd446875b768b72065fde37197f9ab2ef24b0a878fe167e6d1b not found: ID does not exist"
Jul 07 23:47:54.060884 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.060841    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"]
Jul 07 23:47:54.064703 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.064671    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e6351-d89b47547-pgz5p"]
Jul 07 23:47:54.651975 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.651942    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:54.652482 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.652463    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:54.653251 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.653226    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:47:54.973843 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.973749    2578 patch_prober.go:28] interesting pod/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 23:47:54.974000 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:54.973833    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 23:47:55.967461 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:55.964917    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a28e237-d8ad-40ad-a17c-807958349f14" path="/var/lib/kubelet/pods/4a28e237-d8ad-40ad-a17c-807958349f14/volumes"
Jul 07 23:47:56.052769 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:56.052738    2578 generic.go:356] "Generic (PLEG): container finished" podID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerID="13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251" exitCode=0
Jul 07 23:47:56.052951 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:56.052813    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" event={"ID":"1ef61c21-6b32-4677-84e8-67d32e135a16","Type":"ContainerDied","Data":"13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251"}
Jul 07 23:47:56.053373 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:56.053355    2578 scope.go:117] "RemoveContainer" containerID="13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251"
Jul 07 23:47:56.054343 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:56.054320    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:56.054398 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:56.054383    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:56.054448 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:56.054416    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:47:57.059832 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:57.059751    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" event={"ID":"1ef61c21-6b32-4677-84e8-67d32e135a16","Type":"ContainerStarted","Data":"164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29"}
Jul 07 23:47:57.349121 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:57.349029    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:47:57.349283 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:57.349261    2578 patch_prober.go:28] interesting pod/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 07 23:47:57.349349 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:57.349322    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 07 23:47:58.349126 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:58.349092    2578 patch_prober.go:28] interesting pod/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 07 23:47:58.349536 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:58.349161    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 07 23:47:59.116636 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:59.116597    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"]
Jul 07 23:47:59.116888 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:59.116861    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy" containerID="cri-o://159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5" gracePeriod=30
Jul 07 23:47:59.348885 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:59.348847    2578 patch_prober.go:28] interesting pod/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 07 23:47:59.349069 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:47:59.348906    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 07 23:48:00.348910 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:00.348870    2578 patch_prober.go:28] interesting pod/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 07 23:48:00.349426 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:00.348937    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 07 23:48:00.504591 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:00.495081    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"]
Jul 07 23:48:00.504591 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:00.495623    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2" podUID="115c9f0e-65ac-4b93-8b6d-e4571fc99c97" containerName="maas-api" containerID="cri-o://229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247" gracePeriod=30
Jul 07 23:48:00.795035 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:00.795004    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:48:00.917134 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:00.917092    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjvf2\" (UniqueName: \"kubernetes.io/projected/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-kube-api-access-wjvf2\") pod \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") "
Jul 07 23:48:00.917360 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:00.917159    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-maas-api-tls\") pod \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") "
Jul 07 23:48:00.917360 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:00.917283    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-openshift-service-ca\") pod \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\" (UID: \"115c9f0e-65ac-4b93-8b6d-e4571fc99c97\") "
Jul 07 23:48:00.917745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:00.917712    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "115c9f0e-65ac-4b93-8b6d-e4571fc99c97" (UID: "115c9f0e-65ac-4b93-8b6d-e4571fc99c97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:48:00.919431 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:00.919402    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "115c9f0e-65ac-4b93-8b6d-e4571fc99c97" (UID: "115c9f0e-65ac-4b93-8b6d-e4571fc99c97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:48:00.919431 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:00.919419    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-kube-api-access-wjvf2" (OuterVolumeSpecName: "kube-api-access-wjvf2") pod "115c9f0e-65ac-4b93-8b6d-e4571fc99c97" (UID: "115c9f0e-65ac-4b93-8b6d-e4571fc99c97"). InnerVolumeSpecName "kube-api-access-wjvf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:48:01.018153 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.018113    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjvf2\" (UniqueName: \"kubernetes.io/projected/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-kube-api-access-wjvf2\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:01.018153 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.018150    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-maas-api-tls\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:01.018153 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.018159    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/115c9f0e-65ac-4b93-8b6d-e4571fc99c97-openshift-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:01.081793 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.081758    2578 generic.go:356] "Generic (PLEG): container finished" podID="115c9f0e-65ac-4b93-8b6d-e4571fc99c97" containerID="229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247" exitCode=0
Jul 07 23:48:01.081966 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.081807    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2" event={"ID":"115c9f0e-65ac-4b93-8b6d-e4571fc99c97","Type":"ContainerDied","Data":"229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247"}
Jul 07 23:48:01.081966 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.081819    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"
Jul 07 23:48:01.081966 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.081851    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2" event={"ID":"115c9f0e-65ac-4b93-8b6d-e4571fc99c97","Type":"ContainerDied","Data":"b5546898a1549c125a8463d84bb4b622094386249229f5a9137d9670152287d1"}
Jul 07 23:48:01.081966 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.081872    2578 scope.go:117] "RemoveContainer" containerID="229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247"
Jul 07 23:48:01.091757 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.091738    2578 scope.go:117] "RemoveContainer" containerID="229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247"
Jul 07 23:48:01.092058 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:48:01.092034    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247\": container with ID starting with 229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247 not found: ID does not exist" containerID="229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247"
Jul 07 23:48:01.092163 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.092076    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247"} err="failed to get container status \"229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247\": rpc error: code = NotFound desc = could not find container \"229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247\": container with ID starting with 229eb4fb52165809b3714d325fdf4e6382d39a76d3bc301611d4673f8f645247 not found: ID does not exist"
Jul 07 23:48:01.109195 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.109143    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"]
Jul 07 23:48:01.112694 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.112662    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fe5e-78467cc7b4-mmvs2"]
Jul 07 23:48:01.351949 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.351870    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:48:01.352318 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.352142    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:48:01.352789 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.352772    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:48:01.963669 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:01.963638    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="115c9f0e-65ac-4b93-8b6d-e4571fc99c97" path="/var/lib/kubelet/pods/115c9f0e-65ac-4b93-8b6d-e4571fc99c97/volumes"
Jul 07 23:48:04.385433 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.385358    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:48:04.449795 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.449757    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-token\") pod \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") "
Jul 07 23:48:04.449995 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.449818    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-socket\") pod \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") "
Jul 07 23:48:04.449995 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.449851    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvc96\" (UniqueName: \"kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-kube-api-access-vvc96\") pod \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") "
Jul 07 23:48:04.450067 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450027    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istiod-ca-cert\") pod \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") "
Jul 07 23:48:04.450100 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450068    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-credential-socket\") pod \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") "
Jul 07 23:48:04.450143 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450096    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-envoy\") pod \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") "
Jul 07 23:48:04.450143 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450118    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-data\") pod \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") "
Jul 07 23:48:04.450257 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450175    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-podinfo\") pod \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") "
Jul 07 23:48:04.450257 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450175    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" (UID: "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:48:04.450257 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450218    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-certs\") pod \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\" (UID: \"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208\") "
Jul 07 23:48:04.450413 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450333    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" (UID: "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:48:04.450413 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450385    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" (UID: "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:48:04.450663 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450550    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istiod-ca-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:04.450663 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450588    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" (UID: "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:48:04.450663 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450593    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-credential-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:04.450663 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.450631    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:04.451216 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.451192    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-data" (OuterVolumeSpecName: "istio-data") pod "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" (UID: "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:48:04.452166 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.452146    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-token" (OuterVolumeSpecName: "istio-token") pod "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" (UID: "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:48:04.452262 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.452159    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-kube-api-access-vvc96" (OuterVolumeSpecName: "kube-api-access-vvc96") pod "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" (UID: "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208"). InnerVolumeSpecName "kube-api-access-vvc96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:48:04.452734 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.452713    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" (UID: "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:48:04.452818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.452714    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" (UID: "7fd8ce9d-85ff-4330-aae4-ce92b2ab6208"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 23:48:04.551987 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.551941    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:04.551987 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.551980    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvc96\" (UniqueName: \"kubernetes.io/projected/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-kube-api-access-vvc96\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:04.551987 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.551990    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-envoy\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:04.551987 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.551999    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-data\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:04.552282 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.552007    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-istio-podinfo\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:04.552282 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:04.552015    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208-workload-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:05.104347 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.104306    2578 generic.go:356] "Generic (PLEG): container finished" podID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerID="159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5" exitCode=0
Jul 07 23:48:05.104517 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.104392    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" event={"ID":"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208","Type":"ContainerDied","Data":"159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5"}
Jul 07 23:48:05.104517 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.104421    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"
Jul 07 23:48:05.104517 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.104459    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5" event={"ID":"7fd8ce9d-85ff-4330-aae4-ce92b2ab6208","Type":"ContainerDied","Data":"5932f96fbeab0762f74f7b33cae6f6b042b3af1c59be064eefb57b6429a5708b"}
Jul 07 23:48:05.104517 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.104488    2578 scope.go:117] "RemoveContainer" containerID="159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5"
Jul 07 23:48:05.114667 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.114645    2578 scope.go:117] "RemoveContainer" containerID="0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079"
Jul 07 23:48:05.123313 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.123286    2578 scope.go:117] "RemoveContainer" containerID="159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5"
Jul 07 23:48:05.123944 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:48:05.123915    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5\": container with ID starting with 159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5 not found: ID does not exist" containerID="159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5"
Jul 07 23:48:05.124072 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.123950    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5"} err="failed to get container status \"159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5\": rpc error: code = NotFound desc = could not find container \"159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5\": container with ID starting with 159d531e03295d2643f962ab379c9f06d6d11ce276fa3ef384c373ddff99c6f5 not found: ID does not exist"
Jul 07 23:48:05.124072 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.123969    2578 scope.go:117] "RemoveContainer" containerID="0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079"
Jul 07 23:48:05.124250 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:48:05.124228    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079\": container with ID starting with 0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079 not found: ID does not exist" containerID="0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079"
Jul 07 23:48:05.124309 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.124256    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079"} err="failed to get container status \"0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079\": rpc error: code = NotFound desc = could not find container \"0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079\": container with ID starting with 0c8865546515282a76a77c22184bafe72b1d68f9270b2d3c2a3080691b8dc079 not found: ID does not exist"
Jul 07 23:48:05.126464 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.126434    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"]
Jul 07 23:48:05.128992 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.128969    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-6e6351-openshift-default-55bcbf9d9f-l68h5"]
Jul 07 23:48:05.964690 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:05.964658    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" path="/var/lib/kubelet/pods/7fd8ce9d-85ff-4330-aae4-ce92b2ab6208/volumes"
Jul 07 23:48:06.410541 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:06.410495    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"]
Jul 07 23:48:06.410893 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:06.410836    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy" containerID="cri-o://164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29" gracePeriod=30
Jul 07 23:48:07.260258 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.260217    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"]
Jul 07 23:48:07.260737 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.260720    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy"
Jul 07 23:48:07.260779 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.260743    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy"
Jul 07 23:48:07.260779 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.260764    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="115c9f0e-65ac-4b93-8b6d-e4571fc99c97" containerName="maas-api"
Jul 07 23:48:07.260779 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.260773    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="115c9f0e-65ac-4b93-8b6d-e4571fc99c97" containerName="maas-api"
Jul 07 23:48:07.260874 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.260793    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a28e237-d8ad-40ad-a17c-807958349f14" containerName="maas-api"
Jul 07 23:48:07.260874 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.260803    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a28e237-d8ad-40ad-a17c-807958349f14" containerName="maas-api"
Jul 07 23:48:07.260937 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.260907    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="115c9f0e-65ac-4b93-8b6d-e4571fc99c97" containerName="maas-api"
Jul 07 23:48:07.260937 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.260921    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a28e237-d8ad-40ad-a17c-807958349f14" containerName="maas-api"
Jul 07 23:48:07.260937 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.260931    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy"
Jul 07 23:48:07.261019 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.260944    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy"
Jul 07 23:48:07.265423 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.265402    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.267877 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.267857    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-d73a99-openshift-default-dockercfg-kjmtf\""
Jul 07 23:48:07.275655 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.275619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-data\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.275655 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.275669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-credential-socket\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.275882 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.275713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-socket\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.275882 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.275737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-certs\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.275882 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.275812    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-envoy\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.275882 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.275866    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49c43386-214a-42cb-948a-efe13a282df5-istio-podinfo\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.276080 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.275962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49c43386-214a-42cb-948a-efe13a282df5-istiod-ca-cert\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.276080 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.276009    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t6w9\" (UniqueName: \"kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-kube-api-access-9t6w9\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.276080 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.276063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-istio-token\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.276698 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.276674    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"]
Jul 07 23:48:07.376999 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.376965    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-data\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.377191 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.377003    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-credential-socket\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.377191 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.377057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-socket\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.377191 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.377086    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-certs\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.377191 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.377140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-envoy\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.377369 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.377197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49c43386-214a-42cb-948a-efe13a282df5-istio-podinfo\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.377369 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.377239    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49c43386-214a-42cb-948a-efe13a282df5-istiod-ca-cert\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.377369 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.377298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t6w9\" (UniqueName: \"kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-kube-api-access-9t6w9\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.377369 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.377340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-istio-token\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.377512 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.377424    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-credential-socket\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.377705 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.377685    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-data\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.377975 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.377951    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-socket\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.378427 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.378399    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-certs\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.378999 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.378977    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49c43386-214a-42cb-948a-efe13a282df5-istiod-ca-cert\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.381167 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.381143    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-envoy\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.381265 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.381200    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49c43386-214a-42cb-948a-efe13a282df5-istio-podinfo\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.386012 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.385986    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-istio-token\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.386525 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.386506    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t6w9\" (UniqueName: \"kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-kube-api-access-9t6w9\") pod \"e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") " pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.576714 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.576614    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:07.730889 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.730863    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"]
Jul 07 23:48:07.733218 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:48:07.733182    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c43386_214a_42cb_948a_efe13a282df5.slice/crio-e8df9e65c211fdebe2a58c6a33619ef046feed6552e5139e0a0cd860e6d1e6a0 WatchSource:0}: Error finding container e8df9e65c211fdebe2a58c6a33619ef046feed6552e5139e0a0cd860e6d1e6a0: Status 404 returned error can't find the container with id e8df9e65c211fdebe2a58c6a33619ef046feed6552e5139e0a0cd860e6d1e6a0
Jul 07 23:48:07.735984 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.735951    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:07.736105 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.736036    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:07.736170 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:07.736107    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:08.123176 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:08.123141    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" event={"ID":"49c43386-214a-42cb-948a-efe13a282df5","Type":"ContainerStarted","Data":"10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b"}
Jul 07 23:48:08.123395 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:08.123193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" event={"ID":"49c43386-214a-42cb-948a-efe13a282df5","Type":"ContainerStarted","Data":"e8df9e65c211fdebe2a58c6a33619ef046feed6552e5139e0a0cd860e6d1e6a0"}
Jul 07 23:48:08.145638 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:08.145579    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" podStartSLOduration=1.145544357 podStartE2EDuration="1.145544357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:48:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:48:08.143291442 +0000 UTC m=+2072.728860805" watchObservedRunningTime="2026-07-07 23:48:08.145544357 +0000 UTC m=+2072.731113738"
Jul 07 23:48:08.577234 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:08.577142    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:08.582060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:08.582034    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:09.128949 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:09.128912    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:09.129846 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:09.129824    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:10.091404 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.091364    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"]
Jul 07 23:48:10.092065 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.092042    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy"
Jul 07 23:48:10.092153 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.092070    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fd8ce9d-85ff-4330-aae4-ce92b2ab6208" containerName="istio-proxy"
Jul 07 23:48:10.095513 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.095489    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:10.098093 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.098068    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d73a99-serving-cert\""
Jul 07 23:48:10.106072 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.105782    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"]
Jul 07 23:48:10.202343 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.201754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c7da999-7a1b-42ed-b90b-b2b54c295f31-maas-api-tls\") pod \"maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:10.202343 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.201844    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7zxc\" (UniqueName: \"kubernetes.io/projected/5c7da999-7a1b-42ed-b90b-b2b54c295f31-kube-api-access-c7zxc\") pod \"maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:10.202343 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.201881    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c7da999-7a1b-42ed-b90b-b2b54c295f31-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:10.303267 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.303218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c7da999-7a1b-42ed-b90b-b2b54c295f31-maas-api-tls\") pod \"maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:10.303461 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.303283    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7zxc\" (UniqueName: \"kubernetes.io/projected/5c7da999-7a1b-42ed-b90b-b2b54c295f31-kube-api-access-c7zxc\") pod \"maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:10.303461 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.303311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c7da999-7a1b-42ed-b90b-b2b54c295f31-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:10.303461 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:48:10.303392    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-serving-cert: secret "maas-api-e2e-shared-a-d73a99-serving-cert" not found
Jul 07 23:48:10.303628 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:48:10.303469    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7da999-7a1b-42ed-b90b-b2b54c295f31-maas-api-tls podName:5c7da999-7a1b-42ed-b90b-b2b54c295f31 nodeName:}" failed. No retries permitted until 2026-07-07 23:48:10.80344989 +0000 UTC m=+2075.389019229 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5c7da999-7a1b-42ed-b90b-b2b54c295f31-maas-api-tls") pod "maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl" (UID: "5c7da999-7a1b-42ed-b90b-b2b54c295f31") : secret "maas-api-e2e-shared-a-d73a99-serving-cert" not found
Jul 07 23:48:10.304189 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.304162    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c7da999-7a1b-42ed-b90b-b2b54c295f31-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:10.318270 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.318230    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7zxc\" (UniqueName: \"kubernetes.io/projected/5c7da999-7a1b-42ed-b90b-b2b54c295f31-kube-api-access-c7zxc\") pod \"maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:10.808180 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.808133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c7da999-7a1b-42ed-b90b-b2b54c295f31-maas-api-tls\") pod \"maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:10.810724 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:10.810621    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c7da999-7a1b-42ed-b90b-b2b54c295f31-maas-api-tls\") pod \"maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:11.009299 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.009257    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:11.155995 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.155968    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"]
Jul 07 23:48:11.158372 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:48:11.158344    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c7da999_7a1b_42ed_b90b_b2b54c295f31.slice/crio-c97df8f3167c151c5084027f82777cefb1c31b70ed243a50d9c9dfaae94e9d7b WatchSource:0}: Error finding container c97df8f3167c151c5084027f82777cefb1c31b70ed243a50d9c9dfaae94e9d7b: Status 404 returned error can't find the container with id c97df8f3167c151c5084027f82777cefb1c31b70ed243a50d9c9dfaae94e9d7b
Jul 07 23:48:11.689666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.689637    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:48:11.716461 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.716432    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-envoy\") pod \"1ef61c21-6b32-4677-84e8-67d32e135a16\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") "
Jul 07 23:48:11.716461 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.716471    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcxgk\" (UniqueName: \"kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-kube-api-access-wcxgk\") pod \"1ef61c21-6b32-4677-84e8-67d32e135a16\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") "
Jul 07 23:48:11.716729 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.716511    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-certs\") pod \"1ef61c21-6b32-4677-84e8-67d32e135a16\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") "
Jul 07 23:48:11.716729 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.716552    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-socket\") pod \"1ef61c21-6b32-4677-84e8-67d32e135a16\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") "
Jul 07 23:48:11.716729 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.716595    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-credential-socket\") pod \"1ef61c21-6b32-4677-84e8-67d32e135a16\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") "
Jul 07 23:48:11.716729 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.716628    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef61c21-6b32-4677-84e8-67d32e135a16-istiod-ca-cert\") pod \"1ef61c21-6b32-4677-84e8-67d32e135a16\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") "
Jul 07 23:48:11.716729 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.716652    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-data\") pod \"1ef61c21-6b32-4677-84e8-67d32e135a16\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") "
Jul 07 23:48:11.716729 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.716727    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-token\") pod \"1ef61c21-6b32-4677-84e8-67d32e135a16\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") "
Jul 07 23:48:11.717044 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.716822    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-podinfo\") pod \"1ef61c21-6b32-4677-84e8-67d32e135a16\" (UID: \"1ef61c21-6b32-4677-84e8-67d32e135a16\") "
Jul 07 23:48:11.717044 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.716862    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ef61c21-6b32-4677-84e8-67d32e135a16" (UID: "1ef61c21-6b32-4677-84e8-67d32e135a16"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:48:11.717146 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.717028    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ef61c21-6b32-4677-84e8-67d32e135a16" (UID: "1ef61c21-6b32-4677-84e8-67d32e135a16"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:48:11.717146 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.717122    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ef61c21-6b32-4677-84e8-67d32e135a16-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ef61c21-6b32-4677-84e8-67d32e135a16" (UID: "1ef61c21-6b32-4677-84e8-67d32e135a16"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:48:11.717146 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.717135    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:11.717315 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.717154    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-workload-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:11.717490 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.717454    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ef61c21-6b32-4677-84e8-67d32e135a16" (UID: "1ef61c21-6b32-4677-84e8-67d32e135a16"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:48:11.717928 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.717898    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ef61c21-6b32-4677-84e8-67d32e135a16" (UID: "1ef61c21-6b32-4677-84e8-67d32e135a16"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:48:11.719218 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.719191    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ef61c21-6b32-4677-84e8-67d32e135a16" (UID: "1ef61c21-6b32-4677-84e8-67d32e135a16"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 23:48:11.719322 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.719191    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ef61c21-6b32-4677-84e8-67d32e135a16" (UID: "1ef61c21-6b32-4677-84e8-67d32e135a16"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:48:11.719388 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.719344    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ef61c21-6b32-4677-84e8-67d32e135a16" (UID: "1ef61c21-6b32-4677-84e8-67d32e135a16"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:48:11.719460 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.719444    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-kube-api-access-wcxgk" (OuterVolumeSpecName: "kube-api-access-wcxgk") pod "1ef61c21-6b32-4677-84e8-67d32e135a16" (UID: "1ef61c21-6b32-4677-84e8-67d32e135a16"). InnerVolumeSpecName "kube-api-access-wcxgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:48:11.818067 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.818029    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:11.818067 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.818066    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-podinfo\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:11.818067 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.818075    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-envoy\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:11.818313 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.818084    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcxgk\" (UniqueName: \"kubernetes.io/projected/1ef61c21-6b32-4677-84e8-67d32e135a16-kube-api-access-wcxgk\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:11.818313 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.818096    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-credential-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:11.818313 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.818111    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef61c21-6b32-4677-84e8-67d32e135a16-istiod-ca-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:11.818313 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:11.818120    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef61c21-6b32-4677-84e8-67d32e135a16-istio-data\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:48:12.152763 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.152727    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl" event={"ID":"5c7da999-7a1b-42ed-b90b-b2b54c295f31","Type":"ContainerStarted","Data":"a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3"}
Jul 07 23:48:12.152971 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.152778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl" event={"ID":"5c7da999-7a1b-42ed-b90b-b2b54c295f31","Type":"ContainerStarted","Data":"c97df8f3167c151c5084027f82777cefb1c31b70ed243a50d9c9dfaae94e9d7b"}
Jul 07 23:48:12.152971 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.152816    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:12.154589 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.154528    2578 generic.go:356] "Generic (PLEG): container finished" podID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerID="164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29" exitCode=0
Jul 07 23:48:12.154738 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.154596    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"
Jul 07 23:48:12.154738 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.154612    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" event={"ID":"1ef61c21-6b32-4677-84e8-67d32e135a16","Type":"ContainerDied","Data":"164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29"}
Jul 07 23:48:12.154738 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.154664    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv" event={"ID":"1ef61c21-6b32-4677-84e8-67d32e135a16","Type":"ContainerDied","Data":"1facab782388a6a60cf038235c077979718e61fc7e28b144333b2f8bd91545c9"}
Jul 07 23:48:12.154738 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.154687    2578 scope.go:117] "RemoveContainer" containerID="164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29"
Jul 07 23:48:12.164044 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.164022    2578 scope.go:117] "RemoveContainer" containerID="13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251"
Jul 07 23:48:12.171314 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.171262    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl" podStartSLOduration=1.8591242609999998 podStartE2EDuration="2.171243099s" totalImagesPullingTime="312.118838ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:48:12.169372132 +0000 UTC m=+2076.754941494" watchObservedRunningTime="2026-07-07 23:48:12.171243099 +0000 UTC m=+2076.756812460"
Jul 07 23:48:12.173158 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.173135    2578 scope.go:117] "RemoveContainer" containerID="164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29"
Jul 07 23:48:12.173526 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:48:12.173504    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29\": container with ID starting with 164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29 not found: ID does not exist" containerID="164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29"
Jul 07 23:48:12.173592 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.173536    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29"} err="failed to get container status \"164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29\": rpc error: code = NotFound desc = could not find container \"164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29\": container with ID starting with 164e466b8750e630408c46086c106c175bccb25c75305231e62b6413df4dde29 not found: ID does not exist"
Jul 07 23:48:12.173592 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.173555    2578 scope.go:117] "RemoveContainer" containerID="13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251"
Jul 07 23:48:12.173820 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:48:12.173803    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251\": container with ID starting with 13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251 not found: ID does not exist" containerID="13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251"
Jul 07 23:48:12.173902 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.173823    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251"} err="failed to get container status \"13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251\": rpc error: code = NotFound desc = could not find container \"13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251\": container with ID starting with 13a7cb3c483c721c589f7be42b9c75b8ed68fc5fc8683dc3e4b0688851f88251 not found: ID does not exist"
Jul 07 23:48:12.184132 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.184095    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"]
Jul 07 23:48:12.190136 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:12.190102    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b9fe5e-openshift-default-87486c85c-pl8bv"]
Jul 07 23:48:13.963457 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:13.963426    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" path="/var/lib/kubelet/pods/1ef61c21-6b32-4677-84e8-67d32e135a16/volumes"
Jul 07 23:48:15.112360 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.112319    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"]
Jul 07 23:48:15.112975 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.112956    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy"
Jul 07 23:48:15.113044 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.112979    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy"
Jul 07 23:48:15.113140 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.113126    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy"
Jul 07 23:48:15.113186 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.113144    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy"
Jul 07 23:48:15.118120 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.118093    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.120742 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.120721    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-2144c8-openshift-default-dockercfg-gtthq\""
Jul 07 23:48:15.125601 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.125546    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"]
Jul 07 23:48:15.148714 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.148677    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-certs\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.148714 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.148714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-data\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.148977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.148743    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-podinfo\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.148977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.148834    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-credential-socket\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.148977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.148901    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr656\" (UniqueName: \"kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-kube-api-access-rr656\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.148977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.148938    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-envoy\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.148977 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.148963    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-token\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.149204 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.149009    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116e5328-f401-4b74-be69-d67ff2e0ea3f-istiod-ca-cert\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.149204 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.149079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-socket\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.249709 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.249668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr656\" (UniqueName: \"kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-kube-api-access-rr656\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.249709 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.249711    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-envoy\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.249989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.249729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-token\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.249989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.249753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116e5328-f401-4b74-be69-d67ff2e0ea3f-istiod-ca-cert\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.249989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.249792    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-socket\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.249989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.249854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-certs\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.249989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.249881    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-data\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.249989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.249915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-podinfo\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.249989 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.249948    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-credential-socket\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.250360 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.250298    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-certs\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.250360 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.250331    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-socket\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.250468 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.250408    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-credential-socket\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.250577 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.250537    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-data\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.250651 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.250602    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116e5328-f401-4b74-be69-d67ff2e0ea3f-istiod-ca-cert\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.252086 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.252067    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-envoy\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.252522 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.252497    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-podinfo\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.258677 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.258653    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-token\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.258959 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.258937    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr656\" (UniqueName: \"kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-kube-api-access-rr656\") pod \"e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") " pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.431715 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.431682    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:15.579254 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.579223    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"]
Jul 07 23:48:15.581000 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:48:15.580972    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod116e5328_f401_4b74_be69_d67ff2e0ea3f.slice/crio-f3fb7ad077260562ef63e02783db01431234380958f2be8b34d6da64e650820c WatchSource:0}: Error finding container f3fb7ad077260562ef63e02783db01431234380958f2be8b34d6da64e650820c: Status 404 returned error can't find the container with id f3fb7ad077260562ef63e02783db01431234380958f2be8b34d6da64e650820c
Jul 07 23:48:15.584964 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.584901    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:15.585101 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.585080    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:15.595250 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:15.585170    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:16.178593 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:16.178266    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" event={"ID":"116e5328-f401-4b74-be69-d67ff2e0ea3f","Type":"ContainerStarted","Data":"48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9"}
Jul 07 23:48:16.178593 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:16.178336    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" event={"ID":"116e5328-f401-4b74-be69-d67ff2e0ea3f","Type":"ContainerStarted","Data":"f3fb7ad077260562ef63e02783db01431234380958f2be8b34d6da64e650820c"}
Jul 07 23:48:16.204929 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:16.204869    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" podStartSLOduration=1.204852323 podStartE2EDuration="1.204852323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:48:16.201268271 +0000 UTC m=+2080.786837635" watchObservedRunningTime="2026-07-07 23:48:16.204852323 +0000 UTC m=+2080.790421684"
Jul 07 23:48:16.432283 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:16.432179    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:16.437708 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:16.437681    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:17.187519 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:17.187477    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:17.196313 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:17.196260    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:18.166243 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.166211    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:48:18.187218 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.187182    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"]
Jul 07 23:48:18.188060 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.188031    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy"
Jul 07 23:48:18.188526 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.188510    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef61c21-6b32-4677-84e8-67d32e135a16" containerName="istio-proxy"
Jul 07 23:48:18.215182 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.214301    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:18.218662 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.218382    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"]
Jul 07 23:48:18.221459 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.220892    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-2144c8-serving-cert\""
Jul 07 23:48:18.298586 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.296127    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p74qx\" (UniqueName: \"kubernetes.io/projected/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-kube-api-access-p74qx\") pod \"maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:18.298586 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.296329    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:18.298586 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.296362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-maas-api-tls\") pod \"maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:18.397195 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.397144    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:18.397407 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.397211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-maas-api-tls\") pod \"maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:18.397407 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.397301    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p74qx\" (UniqueName: \"kubernetes.io/projected/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-kube-api-access-p74qx\") pod \"maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:18.398059 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:48:18.398032    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-serving-cert: secret "maas-api-e2e-shared-b-2144c8-serving-cert" not found
Jul 07 23:48:18.398155 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:48:18.398112    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-maas-api-tls podName:60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0 nodeName:}" failed. No retries permitted until 2026-07-07 23:48:18.898089916 +0000 UTC m=+2083.483659259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-maas-api-tls") pod "maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr" (UID: "60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0") : secret "maas-api-e2e-shared-b-2144c8-serving-cert" not found
Jul 07 23:48:18.398464 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.398439    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:18.408296 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.408252    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p74qx\" (UniqueName: \"kubernetes.io/projected/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-kube-api-access-p74qx\") pod \"maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:18.903256 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.903213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-maas-api-tls\") pod \"maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:18.905704 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:18.905680    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-maas-api-tls\") pod \"maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:19.131427 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:19.131384    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:19.321335 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:19.321310    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"]
Jul 07 23:48:19.323277 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:48:19.323250    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e42d37_ff1b_4c3c_b1a9_9a8eefedeae0.slice/crio-0036f6534a9c9efb4b24b52a714287cec00dde23ee99da5447202921689863f2 WatchSource:0}: Error finding container 0036f6534a9c9efb4b24b52a714287cec00dde23ee99da5447202921689863f2: Status 404 returned error can't find the container with id 0036f6534a9c9efb4b24b52a714287cec00dde23ee99da5447202921689863f2
Jul 07 23:48:20.230802 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:20.230769    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr" event={"ID":"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0","Type":"ContainerStarted","Data":"0bc60e9ae9f7df83f44e532f5f88226560fc5f6bac9da54219d20bce87b82df3"}
Jul 07 23:48:20.231009 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:20.230821    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr" event={"ID":"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0","Type":"ContainerStarted","Data":"0036f6534a9c9efb4b24b52a714287cec00dde23ee99da5447202921689863f2"}
Jul 07 23:48:20.231009 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:20.230851    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:20.249434 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:20.249374    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr" podStartSLOduration=1.8481252110000002 podStartE2EDuration="2.249357169s" totalImagesPullingTime="401.231958ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:48:20.247133478 +0000 UTC m=+2084.832702840" watchObservedRunningTime="2026-07-07 23:48:20.249357169 +0000 UTC m=+2084.834926531"
Jul 07 23:48:24.130983 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:24.130946    2578 patch_prober.go:28] interesting pod/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 07 23:48:24.131397 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:24.131018    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 07 23:48:24.253855 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:24.253820    2578 generic.go:356] "Generic (PLEG): container finished" podID="49c43386-214a-42cb-948a-efe13a282df5" containerID="10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b" exitCode=0
Jul 07 23:48:24.254030 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:24.253899    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" event={"ID":"49c43386-214a-42cb-948a-efe13a282df5","Type":"ContainerDied","Data":"10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b"}
Jul 07 23:48:24.254313 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:24.254299    2578 scope.go:117] "RemoveContainer" containerID="10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b"
Jul 07 23:48:24.255363 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:24.255339    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:24.255419 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:24.255402    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:24.255455 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:24.255433    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:25.261511 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:25.261481    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" event={"ID":"49c43386-214a-42cb-948a-efe13a282df5","Type":"ContainerStarted","Data":"173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42"}
Jul 07 23:48:25.584782 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:25.577266    2578 patch_prober.go:28] interesting pod/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 07 23:48:25.585316 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:25.576911    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:25.585434 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:25.585083    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 07 23:48:26.242304 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:26.242267    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:48:26.577806 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:26.577702    2578 patch_prober.go:28] interesting pod/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 07 23:48:26.578406 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:26.578372    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 07 23:48:27.577700 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:27.577662    2578 patch_prober.go:28] interesting pod/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 07 23:48:27.578104 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:27.578073    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 07 23:48:28.577930 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:28.577887    2578 patch_prober.go:28] interesting pod/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 07 23:48:28.578222 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:28.578195    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 07 23:48:29.581271 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:29.581191    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:29.581763 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:29.581590    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:29.582414 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:29.582395    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:48:32.189275 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:32.189233    2578 patch_prober.go:28] interesting pod/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 07 23:48:32.189749 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:32.189296    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 07 23:48:32.298869 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:32.298831    2578 generic.go:356] "Generic (PLEG): container finished" podID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerID="48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9" exitCode=0
Jul 07 23:48:32.299043 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:32.298906    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" event={"ID":"116e5328-f401-4b74-be69-d67ff2e0ea3f","Type":"ContainerDied","Data":"48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9"}
Jul 07 23:48:32.299348 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:32.299332    2578 scope.go:117] "RemoveContainer" containerID="48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9"
Jul 07 23:48:32.300300 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:32.300274    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:32.300360 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:32.300338    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:32.300401 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:32.300367    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 23:48:33.307110 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:33.307079    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" event={"ID":"116e5328-f401-4b74-be69-d67ff2e0ea3f","Type":"ContainerStarted","Data":"d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841"}
Jul 07 23:48:33.432149 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:33.432101    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:33.432364 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:33.432339    2578 patch_prober.go:28] interesting pod/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 07 23:48:33.432418 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:33.432388    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 07 23:48:34.432677 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:34.432640    2578 patch_prober.go:28] interesting pod/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 07 23:48:34.433147 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:34.432694    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 07 23:48:35.432293 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:35.432251    2578 patch_prober.go:28] interesting pod/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 07 23:48:35.432488 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:35.432322    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 07 23:48:36.432335 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:36.432290    2578 patch_prober.go:28] interesting pod/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 07 23:48:36.432774 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:36.432353    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 07 23:48:37.436350 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:37.436318    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:37.436756 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:37.436610    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:48:37.437358 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:48:37.437341    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:51:28.166150 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.166070    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"]
Jul 07 23:51:28.166753 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.166305    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl" podUID="5c7da999-7a1b-42ed-b90b-b2b54c295f31" containerName="maas-api" containerID="cri-o://a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3" gracePeriod=30
Jul 07 23:51:28.417592 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.417500    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:51:28.444511 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.444482    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c7da999-7a1b-42ed-b90b-b2b54c295f31-maas-api-tls\") pod \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") "
Jul 07 23:51:28.444720 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.444518    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c7da999-7a1b-42ed-b90b-b2b54c295f31-openshift-service-ca\") pod \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") "
Jul 07 23:51:28.444720 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.444540    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7zxc\" (UniqueName: \"kubernetes.io/projected/5c7da999-7a1b-42ed-b90b-b2b54c295f31-kube-api-access-c7zxc\") pod \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\" (UID: \"5c7da999-7a1b-42ed-b90b-b2b54c295f31\") "
Jul 07 23:51:28.444949 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.444917    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c7da999-7a1b-42ed-b90b-b2b54c295f31-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5c7da999-7a1b-42ed-b90b-b2b54c295f31" (UID: "5c7da999-7a1b-42ed-b90b-b2b54c295f31"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:51:28.446604 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.446583    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c7da999-7a1b-42ed-b90b-b2b54c295f31-kube-api-access-c7zxc" (OuterVolumeSpecName: "kube-api-access-c7zxc") pod "5c7da999-7a1b-42ed-b90b-b2b54c295f31" (UID: "5c7da999-7a1b-42ed-b90b-b2b54c295f31"). InnerVolumeSpecName "kube-api-access-c7zxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:51:28.447078 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.447055    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c7da999-7a1b-42ed-b90b-b2b54c295f31-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5c7da999-7a1b-42ed-b90b-b2b54c295f31" (UID: "5c7da999-7a1b-42ed-b90b-b2b54c295f31"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:51:28.545475 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.545432    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c7da999-7a1b-42ed-b90b-b2b54c295f31-maas-api-tls\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:28.545475 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.545467    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c7da999-7a1b-42ed-b90b-b2b54c295f31-openshift-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:28.545475 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:28.545477    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7zxc\" (UniqueName: \"kubernetes.io/projected/5c7da999-7a1b-42ed-b90b-b2b54c295f31-kube-api-access-c7zxc\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:29.191471 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:29.191430    2578 generic.go:356] "Generic (PLEG): container finished" podID="5c7da999-7a1b-42ed-b90b-b2b54c295f31" containerID="a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3" exitCode=0
Jul 07 23:51:29.191967 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:29.191479    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"
Jul 07 23:51:29.191967 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:29.191520    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl" event={"ID":"5c7da999-7a1b-42ed-b90b-b2b54c295f31","Type":"ContainerDied","Data":"a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3"}
Jul 07 23:51:29.191967 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:29.191612    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl" event={"ID":"5c7da999-7a1b-42ed-b90b-b2b54c295f31","Type":"ContainerDied","Data":"c97df8f3167c151c5084027f82777cefb1c31b70ed243a50d9c9dfaae94e9d7b"}
Jul 07 23:51:29.191967 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:29.191649    2578 scope.go:117] "RemoveContainer" containerID="a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3"
Jul 07 23:51:29.204081 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:29.204056    2578 scope.go:117] "RemoveContainer" containerID="a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3"
Jul 07 23:51:29.204382 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:51:29.204356    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3\": container with ID starting with a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3 not found: ID does not exist" containerID="a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3"
Jul 07 23:51:29.204467 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:29.204392    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3"} err="failed to get container status \"a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3\": rpc error: code = NotFound desc = could not find container \"a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3\": container with ID starting with a144fb304bef0bd643107fc5a4200af351d12a1cfcc7ba912b20140791eae8c3 not found: ID does not exist"
Jul 07 23:51:29.214871 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:29.214845    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"]
Jul 07 23:51:29.219246 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:29.219226    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d73a99-5c6b9fff8c-tp6rl"]
Jul 07 23:51:29.963823 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:29.963785    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c7da999-7a1b-42ed-b90b-b2b54c295f31" path="/var/lib/kubelet/pods/5c7da999-7a1b-42ed-b90b-b2b54c295f31/volumes"
Jul 07 23:51:33.914321 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:33.914284    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"]
Jul 07 23:51:33.914819 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:33.914574    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy" containerID="cri-o://173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42" gracePeriod=30
Jul 07 23:51:35.052309 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.052275    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"]
Jul 07 23:51:35.052819 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.052533    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr" podUID="60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0" containerName="maas-api" containerID="cri-o://0bc60e9ae9f7df83f44e532f5f88226560fc5f6bac9da54219d20bce87b82df3" gracePeriod=30
Jul 07 23:51:35.228290 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.227408    2578 generic.go:356] "Generic (PLEG): container finished" podID="60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0" containerID="0bc60e9ae9f7df83f44e532f5f88226560fc5f6bac9da54219d20bce87b82df3" exitCode=0
Jul 07 23:51:35.228290 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.227500    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr" event={"ID":"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0","Type":"ContainerDied","Data":"0bc60e9ae9f7df83f44e532f5f88226560fc5f6bac9da54219d20bce87b82df3"}
Jul 07 23:51:35.303874 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.303812    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:51:35.410502 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.410473    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-openshift-service-ca\") pod \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") "
Jul 07 23:51:35.410696 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.410528    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p74qx\" (UniqueName: \"kubernetes.io/projected/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-kube-api-access-p74qx\") pod \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") "
Jul 07 23:51:35.410696 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.410620    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-maas-api-tls\") pod \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\" (UID: \"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0\") "
Jul 07 23:51:35.410856 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.410832    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0" (UID: "60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:51:35.410990 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.410970    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-openshift-service-ca\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:35.412707 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.412679    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0" (UID: "60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 23:51:35.412707 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.412697    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-kube-api-access-p74qx" (OuterVolumeSpecName: "kube-api-access-p74qx") pod "60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0" (UID: "60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0"). InnerVolumeSpecName "kube-api-access-p74qx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:51:35.511878 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.511843    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-maas-api-tls\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:35.511878 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:35.511872    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p74qx\" (UniqueName: \"kubernetes.io/projected/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0-kube-api-access-p74qx\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:36.234550 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:36.234464    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr" event={"ID":"60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0","Type":"ContainerDied","Data":"0036f6534a9c9efb4b24b52a714287cec00dde23ee99da5447202921689863f2"}
Jul 07 23:51:36.234550 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:36.234482    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"
Jul 07 23:51:36.234550 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:36.234521    2578 scope.go:117] "RemoveContainer" containerID="0bc60e9ae9f7df83f44e532f5f88226560fc5f6bac9da54219d20bce87b82df3"
Jul 07 23:51:36.258017 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:36.257991    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"]
Jul 07 23:51:36.260736 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:36.260712    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2144c8-684f98cc49-8rpkr"]
Jul 07 23:51:37.962894 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:37.962866    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0" path="/var/lib/kubelet/pods/60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0/volumes"
Jul 07 23:51:39.169796 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.169770    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:51:39.246745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.246648    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49c43386-214a-42cb-948a-efe13a282df5-istio-podinfo\") pod \"49c43386-214a-42cb-948a-efe13a282df5\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") "
Jul 07 23:51:39.246745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.246706    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-data\") pod \"49c43386-214a-42cb-948a-efe13a282df5\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") "
Jul 07 23:51:39.246745 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.246726    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-istio-token\") pod \"49c43386-214a-42cb-948a-efe13a282df5\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") "
Jul 07 23:51:39.247034 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.246771    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49c43386-214a-42cb-948a-efe13a282df5-istiod-ca-cert\") pod \"49c43386-214a-42cb-948a-efe13a282df5\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") "
Jul 07 23:51:39.247034 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.246796    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t6w9\" (UniqueName: \"kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-kube-api-access-9t6w9\") pod \"49c43386-214a-42cb-948a-efe13a282df5\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") "
Jul 07 23:51:39.247034 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.246816    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-credential-socket\") pod \"49c43386-214a-42cb-948a-efe13a282df5\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") "
Jul 07 23:51:39.247034 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.246850    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-socket\") pod \"49c43386-214a-42cb-948a-efe13a282df5\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") "
Jul 07 23:51:39.247034 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.246875    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-envoy\") pod \"49c43386-214a-42cb-948a-efe13a282df5\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") "
Jul 07 23:51:39.247034 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.246950    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-certs\") pod \"49c43386-214a-42cb-948a-efe13a282df5\" (UID: \"49c43386-214a-42cb-948a-efe13a282df5\") "
Jul 07 23:51:39.247331 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.247195    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49c43386-214a-42cb-948a-efe13a282df5" (UID: "49c43386-214a-42cb-948a-efe13a282df5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:51:39.247331 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.247218    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49c43386-214a-42cb-948a-efe13a282df5" (UID: "49c43386-214a-42cb-948a-efe13a282df5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:51:39.247331 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.247301    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-credential-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:39.247331 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.247322    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:39.247547 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.247400    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49c43386-214a-42cb-948a-efe13a282df5" (UID: "49c43386-214a-42cb-948a-efe13a282df5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:51:39.247637 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.247592    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49c43386-214a-42cb-948a-efe13a282df5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49c43386-214a-42cb-948a-efe13a282df5" (UID: "49c43386-214a-42cb-948a-efe13a282df5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:51:39.247930 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.247902    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-data" (OuterVolumeSpecName: "istio-data") pod "49c43386-214a-42cb-948a-efe13a282df5" (UID: "49c43386-214a-42cb-948a-efe13a282df5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:51:39.249207 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.249176    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49c43386-214a-42cb-948a-efe13a282df5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49c43386-214a-42cb-948a-efe13a282df5" (UID: "49c43386-214a-42cb-948a-efe13a282df5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 23:51:39.249414 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.249395    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49c43386-214a-42cb-948a-efe13a282df5" (UID: "49c43386-214a-42cb-948a-efe13a282df5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:51:39.249484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.249420    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-kube-api-access-9t6w9" (OuterVolumeSpecName: "kube-api-access-9t6w9") pod "49c43386-214a-42cb-948a-efe13a282df5" (UID: "49c43386-214a-42cb-948a-efe13a282df5"). InnerVolumeSpecName "kube-api-access-9t6w9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:51:39.249484 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.249448    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-istio-token" (OuterVolumeSpecName: "istio-token") pod "49c43386-214a-42cb-948a-efe13a282df5" (UID: "49c43386-214a-42cb-948a-efe13a282df5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:51:39.254079 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.254049    2578 generic.go:356] "Generic (PLEG): container finished" podID="49c43386-214a-42cb-948a-efe13a282df5" containerID="173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42" exitCode=0
Jul 07 23:51:39.254180 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.254100    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"
Jul 07 23:51:39.254180 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.254136    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" event={"ID":"49c43386-214a-42cb-948a-efe13a282df5","Type":"ContainerDied","Data":"173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42"}
Jul 07 23:51:39.254295 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.254193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2" event={"ID":"49c43386-214a-42cb-948a-efe13a282df5","Type":"ContainerDied","Data":"e8df9e65c211fdebe2a58c6a33619ef046feed6552e5139e0a0cd860e6d1e6a0"}
Jul 07 23:51:39.254295 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.254218    2578 scope.go:117] "RemoveContainer" containerID="173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42"
Jul 07 23:51:39.268847 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.268828    2578 scope.go:117] "RemoveContainer" containerID="10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b"
Jul 07 23:51:39.276848 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.276832    2578 scope.go:117] "RemoveContainer" containerID="173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42"
Jul 07 23:51:39.277092 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:51:39.277076    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42\": container with ID starting with 173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42 not found: ID does not exist" containerID="173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42"
Jul 07 23:51:39.277138 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.277101    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42"} err="failed to get container status \"173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42\": rpc error: code = NotFound desc = could not find container \"173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42\": container with ID starting with 173b36b140342e5897b512209004aebf000f0bdd644a035597812cc1680f3f42 not found: ID does not exist"
Jul 07 23:51:39.277138 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.277117    2578 scope.go:117] "RemoveContainer" containerID="10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b"
Jul 07 23:51:39.277348 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:51:39.277328    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b\": container with ID starting with 10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b not found: ID does not exist" containerID="10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b"
Jul 07 23:51:39.277410 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.277369    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b"} err="failed to get container status \"10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b\": rpc error: code = NotFound desc = could not find container \"10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b\": container with ID starting with 10d908a58f429d579491299a79e59e94b97a16355ff584b6e8227d232ee2b47b not found: ID does not exist"
Jul 07 23:51:39.280519 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.280495    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"]
Jul 07 23:51:39.284535 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.284516    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-d73a99-openshift-default-5f897564c5-m6hc2"]
Jul 07 23:51:39.348076 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.348042    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49c43386-214a-42cb-948a-efe13a282df5-istio-podinfo\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:39.348076 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.348069    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-data\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:39.348076 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.348078    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:39.348076 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.348085    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49c43386-214a-42cb-948a-efe13a282df5-istiod-ca-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:39.348340 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.348094    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t6w9\" (UniqueName: \"kubernetes.io/projected/49c43386-214a-42cb-948a-efe13a282df5-kube-api-access-9t6w9\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:39.348340 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.348104    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-istio-envoy\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:39.348340 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.348113    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49c43386-214a-42cb-948a-efe13a282df5-workload-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:39.963233 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:39.963196    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49c43386-214a-42cb-948a-efe13a282df5" path="/var/lib/kubelet/pods/49c43386-214a-42cb-948a-efe13a282df5/volumes"
Jul 07 23:51:40.875126 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:40.875093    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"]
Jul 07 23:51:40.875610 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:40.875417    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy" containerID="cri-o://d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841" gracePeriod=30
Jul 07 23:51:46.138141 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.138072    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:51:46.212391 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212362    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-certs\") pod \"116e5328-f401-4b74-be69-d67ff2e0ea3f\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") "
Jul 07 23:51:46.212594 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212417    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116e5328-f401-4b74-be69-d67ff2e0ea3f-istiod-ca-cert\") pod \"116e5328-f401-4b74-be69-d67ff2e0ea3f\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") "
Jul 07 23:51:46.212594 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212450    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-data\") pod \"116e5328-f401-4b74-be69-d67ff2e0ea3f\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") "
Jul 07 23:51:46.212594 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212489    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-envoy\") pod \"116e5328-f401-4b74-be69-d67ff2e0ea3f\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") "
Jul 07 23:51:46.212594 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212516    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-podinfo\") pod \"116e5328-f401-4b74-be69-d67ff2e0ea3f\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") "
Jul 07 23:51:46.212818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212595    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-socket\") pod \"116e5328-f401-4b74-be69-d67ff2e0ea3f\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") "
Jul 07 23:51:46.212818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212641    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-credential-socket\") pod \"116e5328-f401-4b74-be69-d67ff2e0ea3f\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") "
Jul 07 23:51:46.212818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212671    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-token\") pod \"116e5328-f401-4b74-be69-d67ff2e0ea3f\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") "
Jul 07 23:51:46.212818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212689    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "116e5328-f401-4b74-be69-d67ff2e0ea3f" (UID: "116e5328-f401-4b74-be69-d67ff2e0ea3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:51:46.212818 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212752    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rr656\" (UniqueName: \"kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-kube-api-access-rr656\") pod \"116e5328-f401-4b74-be69-d67ff2e0ea3f\" (UID: \"116e5328-f401-4b74-be69-d67ff2e0ea3f\") "
Jul 07 23:51:46.213063 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212860    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/116e5328-f401-4b74-be69-d67ff2e0ea3f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "116e5328-f401-4b74-be69-d67ff2e0ea3f" (UID: "116e5328-f401-4b74-be69-d67ff2e0ea3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 23:51:46.213063 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.212928    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "116e5328-f401-4b74-be69-d67ff2e0ea3f" (UID: "116e5328-f401-4b74-be69-d67ff2e0ea3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:51:46.213180 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.213095    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "116e5328-f401-4b74-be69-d67ff2e0ea3f" (UID: "116e5328-f401-4b74-be69-d67ff2e0ea3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:51:46.213180 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.213107    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:46.213180 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.213137    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-workload-certs\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:46.213180 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.213154    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116e5328-f401-4b74-be69-d67ff2e0ea3f-istiod-ca-cert\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:46.213466 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.213442    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-data" (OuterVolumeSpecName: "istio-data") pod "116e5328-f401-4b74-be69-d67ff2e0ea3f" (UID: "116e5328-f401-4b74-be69-d67ff2e0ea3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:51:46.214996 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.214973    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-token" (OuterVolumeSpecName: "istio-token") pod "116e5328-f401-4b74-be69-d67ff2e0ea3f" (UID: "116e5328-f401-4b74-be69-d67ff2e0ea3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:51:46.215141 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.215030    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-kube-api-access-rr656" (OuterVolumeSpecName: "kube-api-access-rr656") pod "116e5328-f401-4b74-be69-d67ff2e0ea3f" (UID: "116e5328-f401-4b74-be69-d67ff2e0ea3f"). InnerVolumeSpecName "kube-api-access-rr656". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 23:51:46.215141 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.215112    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "116e5328-f401-4b74-be69-d67ff2e0ea3f" (UID: "116e5328-f401-4b74-be69-d67ff2e0ea3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 23:51:46.215255 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.215188    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "116e5328-f401-4b74-be69-d67ff2e0ea3f" (UID: "116e5328-f401-4b74-be69-d67ff2e0ea3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 23:51:46.239122 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.239086    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2wdx7_444865b2-ad2c-43e3-959e-a8bc5573234e/manager/0.log"
Jul 07 23:51:46.292444 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.292407    2578 generic.go:356] "Generic (PLEG): container finished" podID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerID="d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841" exitCode=0
Jul 07 23:51:46.292668 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.292462    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"
Jul 07 23:51:46.292668 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.292491    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" event={"ID":"116e5328-f401-4b74-be69-d67ff2e0ea3f","Type":"ContainerDied","Data":"d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841"}
Jul 07 23:51:46.292668 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.292543    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9" event={"ID":"116e5328-f401-4b74-be69-d67ff2e0ea3f","Type":"ContainerDied","Data":"f3fb7ad077260562ef63e02783db01431234380958f2be8b34d6da64e650820c"}
Jul 07 23:51:46.292668 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.292603    2578 scope.go:117] "RemoveContainer" containerID="d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841"
Jul 07 23:51:46.302077 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.302058    2578 scope.go:117] "RemoveContainer" containerID="48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9"
Jul 07 23:51:46.309999 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.309982    2578 scope.go:117] "RemoveContainer" containerID="d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841"
Jul 07 23:51:46.310251 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:51:46.310232    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841\": container with ID starting with d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841 not found: ID does not exist" containerID="d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841"
Jul 07 23:51:46.310319 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.310259    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841"} err="failed to get container status \"d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841\": rpc error: code = NotFound desc = could not find container \"d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841\": container with ID starting with d8c6fb608199497f7957ad868ff2242fa349f6e3a4aa21fc585cf7a5c1f1d841 not found: ID does not exist"
Jul 07 23:51:46.310319 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.310279    2578 scope.go:117] "RemoveContainer" containerID="48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9"
Jul 07 23:51:46.310509 ip-10-0-137-77 kubenswrapper[2578]: E0707 23:51:46.310491    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9\": container with ID starting with 48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9 not found: ID does not exist" containerID="48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9"
Jul 07 23:51:46.310551 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.310517    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9"} err="failed to get container status \"48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9\": rpc error: code = NotFound desc = could not find container \"48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9\": container with ID starting with 48ecb821192ea843f2be9e746ee591928e0abd1fda86c7e18e0dcd66a52370d9 not found: ID does not exist"
Jul 07 23:51:46.314304 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.314286    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-credential-socket\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:46.314371 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.314307    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-token\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:46.314371 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.314317    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr656\" (UniqueName: \"kubernetes.io/projected/116e5328-f401-4b74-be69-d67ff2e0ea3f-kube-api-access-rr656\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:46.314371 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.314325    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-data\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:46.314371 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.314333    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-envoy\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:46.314371 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.314341    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116e5328-f401-4b74-be69-d67ff2e0ea3f-istio-podinfo\") on node \"ip-10-0-137-77.ec2.internal\" DevicePath \"\""
Jul 07 23:51:46.326150 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.326120    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"]
Jul 07 23:51:46.332143 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.332121    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-2144c8-openshift-default-77dc589b87-6v5h9"]
Jul 07 23:51:46.373282 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.373253    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-k87dr_b2762acc-d7af-4ca5-817e-f201d470d0ef/manager/0.log"
Jul 07 23:51:46.502240 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.502156    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-85dc9d69dd-6ts5f_ecad7d8d-213a-485b-95f8-b0b809d04dce/manager/0.log"
Jul 07 23:51:46.616840 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.616806    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vlkvz_719d2eb5-c1d4-45bf-adc5-dd16f0328209/server/0.log"
Jul 07 23:51:46.740866 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.740839    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jk5xl_312d941a-d76a-4cb8-aa34-7b4d5271bd20/manager/0.log"
Jul 07 23:51:46.988953 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.988925    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9859966d-f6dzb_4d880249-e1c5-4c06-95b0-3fc875533f36/copy-manifests/0.log"
Jul 07 23:51:46.996625 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:46.996598    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9859966d-f6dzb_4d880249-e1c5-4c06-95b0-3fc875533f36/manager/0.log"
Jul 07 23:51:47.962941 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:47.962911    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" path="/var/lib/kubelet/pods/116e5328-f401-4b74-be69-d67ff2e0ea3f/volumes"
Jul 07 23:51:48.563487 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:48.563455    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hhf48_f7aae87a-c894-43c1-9426-d164ec5c391a/manager/0.log"
Jul 07 23:51:48.673303 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:48.673275    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hpnqt_de987442-d98a-403c-9a1b-ed2a61c2fd02/manager/0.log"
Jul 07 23:51:48.913669 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:48.913636    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h96lz_bd112ae1-1905-4b0e-9af2-6a4d955ef92c/registry-server/0.log"
Jul 07 23:51:49.268213 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:49.268125    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4dkdc_0e26a7d9-fd80-49b0-af32-35d646136692/manager/0.log"
Jul 07 23:51:49.614633 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:49.614542    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786np6t_ea88039b-9e95-4ccc-abeb-80a7f2fa2f72/istio-proxy/0.log"
Jul 07 23:51:50.905398 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:50.905352    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b7668c465-5rrp6_5abf2439-622a-4389-b746-d87f33ea9c7d/kube-auth-proxy/0.log"
Jul 07 23:51:51.141028 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:51.140998    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-sdsln_60a067ee-b2ba-469f-8b34-a6af603e638e/istio-proxy/1.log"
Jul 07 23:51:51.405094 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:51.405054    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rngq2_9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb/payload-processing/0.log"
Jul 07 23:51:51.859736 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:51.859640    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-9jdmp_4ea3894c-863a-435d-86c1-81120d83265c/main/0.log"
Jul 07 23:51:52.090299 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:52.090265    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-9qsxw_b61808c5-1ac4-465b-8ae9-0ec8d3485cff/main/0.log"
Jul 07 23:51:59.759832 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:59.759799    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5kxrk_5f0cc873-6ed5-478d-b3b9-f563d8f8b773/global-pull-secret-syncer/0.log"
Jul 07 23:51:59.957879 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:51:59.957843    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wv855_622e9231-8a5d-4998-a298-d76e51084e83/konnectivity-agent/0.log"
Jul 07 23:52:00.062075 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:00.061987    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-77.ec2.internal_f393c9727e900c86fef1db62fb4e666a/haproxy/0.log"
Jul 07 23:52:05.819713 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:05.819678    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hhf48_f7aae87a-c894-43c1-9426-d164ec5c391a/manager/0.log"
Jul 07 23:52:05.844810 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:05.844774    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hpnqt_de987442-d98a-403c-9a1b-ed2a61c2fd02/manager/0.log"
Jul 07 23:52:05.929308 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:05.929266    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h96lz_bd112ae1-1905-4b0e-9af2-6a4d955ef92c/registry-server/0.log"
Jul 07 23:52:06.180468 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:06.180437    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4dkdc_0e26a7d9-fd80-49b0-af32-35d646136692/manager/0.log"
Jul 07 23:52:07.967908 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:07.967875    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-llz4m_879d5116-7b20-4d96-8406-8b4f05358e48/node-exporter/0.log"
Jul 07 23:52:07.988275 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:07.988249    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-llz4m_879d5116-7b20-4d96-8406-8b4f05358e48/kube-rbac-proxy/0.log"
Jul 07 23:52:08.012008 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:08.011983    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-llz4m_879d5116-7b20-4d96-8406-8b4f05358e48/init-textfile/0.log"
Jul 07 23:52:09.977043 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977011    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"]
Jul 07 23:52:09.977425 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977388    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy"
Jul 07 23:52:09.977425 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977400    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy"
Jul 07 23:52:09.977425 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977408    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy"
Jul 07 23:52:09.977425 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977414    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy"
Jul 07 23:52:09.977425 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977420    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy"
Jul 07 23:52:09.977425 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977426    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977445    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977451    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977459    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0" containerName="maas-api"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977464    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0" containerName="maas-api"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977478    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c7da999-7a1b-42ed-b90b-b2b54c295f31" containerName="maas-api"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977483    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c7da999-7a1b-42ed-b90b-b2b54c295f31" containerName="maas-api"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977533    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977540    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c7da999-7a1b-42ed-b90b-b2b54c295f31" containerName="maas-api"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977549    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977555    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="49c43386-214a-42cb-948a-efe13a282df5" containerName="istio-proxy"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977577    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e42d37-ff1b-4c3c-b1a9-9a8eefedeae0" containerName="maas-api"
Jul 07 23:52:09.977666 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.977585    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="116e5328-f401-4b74-be69-d67ff2e0ea3f" containerName="istio-proxy"
Jul 07 23:52:09.982130 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.982110    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:09.984861 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.984840    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dx64r\"/\"kube-root-ca.crt\""
Jul 07 23:52:09.985940 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.985914    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dx64r\"/\"openshift-service-ca.crt\""
Jul 07 23:52:09.986069 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.985962    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dx64r\"/\"default-dockercfg-52fsr\""
Jul 07 23:52:09.988091 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:09.988069    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"]
Jul 07 23:52:10.020699 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.020662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-podres\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.020896 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.020788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-lib-modules\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.020896 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.020843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-proc\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.020994 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.020927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-sys\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.021075 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.021053    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2cpj\" (UniqueName: \"kubernetes.io/projected/b8385516-b358-45bc-a4e9-ef29d03ec5db-kube-api-access-c2cpj\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.121819 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.121782    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-podres\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.122021 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.121829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-lib-modules\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.122021 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.121864    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-proc\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.122021 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.121903    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-sys\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.122021 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.121965    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2cpj\" (UniqueName: \"kubernetes.io/projected/b8385516-b358-45bc-a4e9-ef29d03ec5db-kube-api-access-c2cpj\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.122021 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.121967    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-podres\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.122021 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.121990    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-proc\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.122021 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.122000    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-lib-modules\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.122308 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.122057    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8385516-b358-45bc-a4e9-ef29d03ec5db-sys\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.131372 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.131348    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2cpj\" (UniqueName: \"kubernetes.io/projected/b8385516-b358-45bc-a4e9-ef29d03ec5db-kube-api-access-c2cpj\") pod \"perf-node-gather-daemonset-dl5nm\" (UID: \"b8385516-b358-45bc-a4e9-ef29d03ec5db\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.292995 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.292895    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:10.433000 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.432940    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"]
Jul 07 23:52:10.435299 ip-10-0-137-77 kubenswrapper[2578]: W0707 23:52:10.435271    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb8385516_b358_45bc_a4e9_ef29d03ec5db.slice/crio-d1f59abb5d462a2ebe6dad0555429068aad16f331a6db47ae55369c25786f1cd WatchSource:0}: Error finding container d1f59abb5d462a2ebe6dad0555429068aad16f331a6db47ae55369c25786f1cd: Status 404 returned error can't find the container with id d1f59abb5d462a2ebe6dad0555429068aad16f331a6db47ae55369c25786f1cd
Jul 07 23:52:10.938086 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:10.938059    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69745cc477-mf458_312bb119-7af4-4263-9d75-f5131a4ae8ee/console/0.log"
Jul 07 23:52:11.421392 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:11.421348    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm" event={"ID":"b8385516-b358-45bc-a4e9-ef29d03ec5db","Type":"ContainerStarted","Data":"a9ddca7786a3a84d728d500e3964187752ccd858b91a2d30f25bb476ff2f555d"}
Jul 07 23:52:11.421800 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:11.421415    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:11.421800 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:11.421439    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm" event={"ID":"b8385516-b358-45bc-a4e9-ef29d03ec5db","Type":"ContainerStarted","Data":"d1f59abb5d462a2ebe6dad0555429068aad16f331a6db47ae55369c25786f1cd"}
Jul 07 23:52:11.450858 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:11.450805    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm" podStartSLOduration=2.450787266 podStartE2EDuration="2.450787266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:52:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:52:11.448989234 +0000 UTC m=+2316.034558595" watchObservedRunningTime="2026-07-07 23:52:11.450787266 +0000 UTC m=+2316.036356680"
Jul 07 23:52:12.643756 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:12.643725    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6vvnp_3f32a465-635f-48b6-b22a-f4e5d481db0b/dns/0.log"
Jul 07 23:52:12.666716 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:12.666678    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6vvnp_3f32a465-635f-48b6-b22a-f4e5d481db0b/kube-rbac-proxy/0.log"
Jul 07 23:52:12.741154 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:12.741121    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7kqxl_f1a6d6ed-76dd-45b7-bfc7-65f0babd3893/dns-node-resolver/0.log"
Jul 07 23:52:13.400165 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:13.400133    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g7tzd_c342d1b2-9987-404d-88ae-197de63d7d04/node-ca/0.log"
Jul 07 23:52:14.359021 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:14.358990    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786np6t_ea88039b-9e95-4ccc-abeb-80a7f2fa2f72/istio-proxy/0.log"
Jul 07 23:52:14.552498 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:14.552470    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b7668c465-5rrp6_5abf2439-622a-4389-b746-d87f33ea9c7d/kube-auth-proxy/0.log"
Jul 07 23:52:14.615788 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:14.615699    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-sdsln_60a067ee-b2ba-469f-8b34-a6af603e638e/istio-proxy/0.log"
Jul 07 23:52:14.668686 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:14.668651    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-sdsln_60a067ee-b2ba-469f-8b34-a6af603e638e/istio-proxy/1.log"
Jul 07 23:52:14.797302 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:14.797273    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rngq2_9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb/payload-processing/0.log"
Jul 07 23:52:15.483215 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:15.483186    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7cbhm_13d513a6-1f84-482a-985a-80339590cd66/serve-healthcheck-canary/0.log"
Jul 07 23:52:16.060856 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:16.060825    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m6fb4_7731486d-cda9-438c-8852-58ff55acb031/kube-rbac-proxy/0.log"
Jul 07 23:52:16.082756 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:16.082728    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m6fb4_7731486d-cda9-438c-8852-58ff55acb031/exporter/0.log"
Jul 07 23:52:16.105327 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:16.105300    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m6fb4_7731486d-cda9-438c-8852-58ff55acb031/extractor/0.log"
Jul 07 23:52:17.435860 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:17.435833    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-dl5nm"
Jul 07 23:52:18.476508 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:18.476478    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2wdx7_444865b2-ad2c-43e3-959e-a8bc5573234e/manager/0.log"
Jul 07 23:52:18.670639 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:18.670603    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-k87dr_b2762acc-d7af-4ca5-817e-f201d470d0ef/manager/0.log"
Jul 07 23:52:18.813189 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:18.813103    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-85dc9d69dd-6ts5f_ecad7d8d-213a-485b-95f8-b0b809d04dce/manager/0.log"
Jul 07 23:52:18.834941 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:18.834913    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vlkvz_719d2eb5-c1d4-45bf-adc5-dd16f0328209/server/0.log"
Jul 07 23:52:18.932326 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:18.932299    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jk5xl_312d941a-d76a-4cb8-aa34-7b4d5271bd20/manager/0.log"
Jul 07 23:52:19.084403 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:19.084323    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9859966d-f6dzb_4d880249-e1c5-4c06-95b0-3fc875533f36/manager/0.log"
Jul 07 23:52:19.110120 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:19.110091    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9859966d-f6dzb_4d880249-e1c5-4c06-95b0-3fc875533f36/copy-manifests/0.log"
Jul 07 23:52:19.915158 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:19.915123    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786np6t_ea88039b-9e95-4ccc-abeb-80a7f2fa2f72/istio-proxy/0.log"
Jul 07 23:52:20.086955 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:20.086928    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b7668c465-5rrp6_5abf2439-622a-4389-b746-d87f33ea9c7d/kube-auth-proxy/0.log"
Jul 07 23:52:20.128231 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:20.128199    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-sdsln_60a067ee-b2ba-469f-8b34-a6af603e638e/istio-proxy/0.log"
Jul 07 23:52:20.184418 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:20.184386    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-sdsln_60a067ee-b2ba-469f-8b34-a6af603e638e/istio-proxy/1.log"
Jul 07 23:52:20.292041 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:20.292008    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rngq2_9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb/payload-processing/0.log"
Jul 07 23:52:21.508862 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:21.508830    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786np6t_ea88039b-9e95-4ccc-abeb-80a7f2fa2f72/istio-proxy/0.log"
Jul 07 23:52:21.684137 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:21.684102    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b7668c465-5rrp6_5abf2439-622a-4389-b746-d87f33ea9c7d/kube-auth-proxy/0.log"
Jul 07 23:52:21.734012 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:21.733981    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-sdsln_60a067ee-b2ba-469f-8b34-a6af603e638e/istio-proxy/0.log"
Jul 07 23:52:21.788149 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:21.788124    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-sdsln_60a067ee-b2ba-469f-8b34-a6af603e638e/istio-proxy/1.log"
Jul 07 23:52:21.897814 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:21.897780    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rngq2_9fb3b4fe-707e-41f6-a6a7-ded7d2c233cb/payload-processing/0.log"
Jul 07 23:52:25.450506 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:25.450469    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-sphqg_e2fdeee5-357d-48fb-b836-08d642f4e852/migrator/0.log"
Jul 07 23:52:25.472898 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:25.472866    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-sphqg_e2fdeee5-357d-48fb-b836-08d642f4e852/graceful-termination/0.log"
Jul 07 23:52:25.823394 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:25.823289    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wlvx5_6a35c17c-c144-4083-b554-15f80a2261b7/kube-storage-version-migrator-operator/1.log"
Jul 07 23:52:25.824895 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:25.824861    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wlvx5_6a35c17c-c144-4083-b554-15f80a2261b7/kube-storage-version-migrator-operator/0.log"
Jul 07 23:52:26.867973 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:26.867941    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxn5_337b8d66-22e8-4cee-a345-c5dd54dca37b/kube-multus-additional-cni-plugins/0.log"
Jul 07 23:52:26.890719 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:26.890688    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxn5_337b8d66-22e8-4cee-a345-c5dd54dca37b/egress-router-binary-copy/0.log"
Jul 07 23:52:26.911682 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:26.911652    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxn5_337b8d66-22e8-4cee-a345-c5dd54dca37b/cni-plugins/0.log"
Jul 07 23:52:26.933268 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:26.933241    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxn5_337b8d66-22e8-4cee-a345-c5dd54dca37b/bond-cni-plugin/0.log"
Jul 07 23:52:26.955165 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:26.955137    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxn5_337b8d66-22e8-4cee-a345-c5dd54dca37b/routeoverride-cni/0.log"
Jul 07 23:52:26.978940 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:26.978912    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxn5_337b8d66-22e8-4cee-a345-c5dd54dca37b/whereabouts-cni-bincopy/0.log"
Jul 07 23:52:27.000309 ip-10-0-137-77 kubenswrapper[2578]: I0707 23:52:27.000279    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxn5_337b8d66-22e8-4cee-a345-c5dd54dca37b/whereabouts-cni/0.log"