Jul 15 04:06:10.569423 ip-10-0-129-76 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 04:06:11.160982 ip-10-0-129-76 kubenswrapper[2524]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 04:06:11.160982 ip-10-0-129-76 kubenswrapper[2524]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 04:06:11.160982 ip-10-0-129-76 kubenswrapper[2524]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 04:06:11.160982 ip-10-0-129-76 kubenswrapper[2524]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 04:06:11.160982 ip-10-0-129-76 kubenswrapper[2524]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 04:06:11.163698 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.163538    2524 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 04:06:11.168238 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168223    2524 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:06:11.168238 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168238    2524 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168249    2524 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168253    2524 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168256    2524 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168258    2524 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168262    2524 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168264    2524 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168267    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168270    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168272    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168275    2524 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168278    2524 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168280    2524 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168283    2524 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168286    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168288    2524 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168291    2524 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168294    2524 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168296    2524 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168299    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:06:11.168307 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168302    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168304    2524 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168307    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168309    2524 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168312    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168315    2524 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168317    2524 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168320    2524 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168323    2524 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168325    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168328    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168330    2524 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168332    2524 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168335    2524 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168338    2524 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168341    2524 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168345    2524 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168347    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168349    2524 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168352    2524 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:06:11.168812 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168354    2524 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168357    2524 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168359    2524 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168362    2524 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168364    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168367    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168370    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168373    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168375    2524 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168378    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168380    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168385    2524 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168389    2524 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168392    2524 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168395    2524 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168397    2524 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168400    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168402    2524 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168406    2524 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:06:11.169289 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168410    2524 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168413    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168415    2524 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168418    2524 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168420    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168423    2524 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168425    2524 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168428    2524 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168431    2524 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168434    2524 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168436    2524 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168439    2524 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168441    2524 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168445    2524 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168449    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168451    2524 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168454    2524 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168457    2524 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168460    2524 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:06:11.169894 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168463    2524 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168466    2524 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168469    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168472    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168474    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168477    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168479    2524 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168483    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168487    2524 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168489    2524 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168492    2524 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168495    2524 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168497    2524 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168500    2524 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168502    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168505    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168508    2524 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168510    2524 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168513    2524 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.168516    2524 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:06:11.170570 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169257    2524 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169264    2524 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169268    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169271    2524 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169273    2524 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169276    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169279    2524 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169281    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169284    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169287    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169290    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169293    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169296    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169299    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169302    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169305    2524 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169307    2524 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169310    2524 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169313    2524 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169315    2524 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:06:11.171065 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169319    2524 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169322    2524 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169324    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169327    2524 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169329    2524 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169332    2524 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169334    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169337    2524 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169339    2524 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169342    2524 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169344    2524 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169347    2524 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169350    2524 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169353    2524 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169356    2524 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169359    2524 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169362    2524 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169365    2524 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169367    2524 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169370    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:06:11.171589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169372    2524 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169375    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169377    2524 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169380    2524 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169383    2524 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169385    2524 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169390    2524 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169393    2524 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169396    2524 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169399    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169403    2524 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169406    2524 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169409    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169412    2524 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169415    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169418    2524 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169422    2524 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169425    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169429    2524 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:06:11.172081 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169431    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169434    2524 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169436    2524 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169439    2524 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169442    2524 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169444    2524 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169447    2524 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169449    2524 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169452    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169506    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169784    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169793    2524 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169798    2524 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169804    2524 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169808    2524 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169814    2524 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169819    2524 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169824    2524 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169829    2524 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:06:11.172531 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169839    2524 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169845    2524 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169851    2524 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169861    2524 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169865    2524 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169869    2524 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169874    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169879    2524 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169884    2524 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169888    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169893    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169897    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169902    2524 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169910    2524 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169914    2524 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169923    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169928    2524 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169932    2524 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169936    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169941    2524 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:06:11.173007 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.169945    2524 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170941    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170951    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170954    2524 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170957    2524 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170960    2524 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170963    2524 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170966    2524 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170968    2524 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170971    2524 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170973    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170976    2524 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170978    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170981    2524 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170984    2524 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170986    2524 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170989    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170991    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170995    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.170997    2524 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:06:11.173487 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171000    2524 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171003    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171005    2524 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171008    2524 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171010    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171013    2524 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171015    2524 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171017    2524 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171020    2524 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171023    2524 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171025    2524 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171027    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171030    2524 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171032    2524 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171036    2524 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171039    2524 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171042    2524 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171045    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171047    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171051    2524 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:06:11.173977 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171055    2524 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171059    2524 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171063    2524 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171066    2524 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171069    2524 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171072    2524 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171075    2524 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171078    2524 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171081    2524 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171084    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171087    2524 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171089    2524 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171093    2524 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171095    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171099    2524 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171101    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171105    2524 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171108    2524 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171110    2524 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:06:11.174443 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171113    2524 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171115    2524 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171118    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171121    2524 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171124    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171126    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171129    2524 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171132    2524 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171135    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171138    2524 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171141    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171143    2524 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171148    2524 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171151    2524 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171154    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171157    2524 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171159    2524 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171162    2524 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171165    2524 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171168    2524 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:06:11.174906 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171170    2524 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171173    2524 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171175    2524 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171178    2524 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171181    2524 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171183    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171186    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171189    2524 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171191    2524 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171194    2524 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171197    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171199    2524 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171202    2524 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171204    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171207    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171209    2524 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171212    2524 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171214    2524 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171216    2524 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171222    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:06:11.175364 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171224    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171294    2524 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171301    2524 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171308    2524 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171313    2524 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171317    2524 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171320    2524 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171324    2524 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171329    2524 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171332    2524 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171335    2524 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171338    2524 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171342    2524 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171345    2524 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171348    2524 flags.go:64] FLAG: --cgroup-root=""
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171351    2524 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171354    2524 flags.go:64] FLAG: --client-ca-file=""
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171357    2524 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171360    2524 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171364    2524 flags.go:64] FLAG: --cluster-domain=""
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171367    2524 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171370    2524 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171373    2524 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171377    2524 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 04:06:11.175852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171380    2524 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171383    2524 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171386    2524 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171389    2524 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171392    2524 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171395    2524 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171398    2524 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171404    2524 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171407    2524 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171411    2524 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171414    2524 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171417    2524 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171420    2524 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171423    2524 flags.go:64] FLAG: --enable-server="true"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171426    2524 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171430    2524 flags.go:64] FLAG: --event-burst="100"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171434    2524 flags.go:64] FLAG: --event-qps="50"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171436    2524 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171439    2524 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171442    2524 flags.go:64] FLAG: --eviction-hard=""
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171446    2524 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171448    2524 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171452    2524 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171455    2524 flags.go:64] FLAG: --eviction-soft=""
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171458    2524 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 04:06:11.176406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171461    2524 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171464    2524 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171467    2524 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171470    2524 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171473    2524 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171476    2524 flags.go:64] FLAG: --feature-gates=""
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171480    2524 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171483    2524 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171486    2524 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171489    2524 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171492    2524 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171495    2524 flags.go:64] FLAG: --help="false"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171498    2524 flags.go:64] FLAG: --hostname-override="ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171501    2524 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171505    2524 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171508    2524 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171511    2524 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171515    2524 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171518    2524 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171520    2524 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171523    2524 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171526    2524 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171529    2524 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171532    2524 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 04:06:11.177062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171535    2524 flags.go:64] FLAG: --kube-reserved=""
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171538    2524 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171540    2524 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171543    2524 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171546    2524 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171549    2524 flags.go:64] FLAG: --lock-file=""
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171551    2524 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171554    2524 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171557    2524 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171563    2524 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171566    2524 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171568    2524 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171572    2524 flags.go:64] FLAG: --logging-format="text"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171574    2524 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171578    2524 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171581    2524 flags.go:64] FLAG: --manifest-url=""
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171583    2524 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171588    2524 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171608    2524 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171614    2524 flags.go:64] FLAG: --max-pods="110"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171617    2524 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171620    2524 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171624    2524 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171627    2524 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171630    2524 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 04:06:11.177660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171634    2524 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171636    2524 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171642    2524 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171645    2524 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171648    2524 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171651    2524 flags.go:64] FLAG: --pod-cidr=""
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171654    2524 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171660    2524 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171663    2524 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171666    2524 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171669    2524 flags.go:64] FLAG: --port="10250"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171672    2524 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171675    2524 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0612bb46ad5d62ae1"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171678    2524 flags.go:64] FLAG: --qos-reserved=""
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171681    2524 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171684    2524 flags.go:64] FLAG: --register-node="true"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171687    2524 flags.go:64] FLAG: --register-with-taints=""
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171690    2524 flags.go:64] FLAG: --registry-burst="10"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171693    2524 flags.go:64] FLAG: --registry-qps="5"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171696    2524 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171699    2524 flags.go:64] FLAG: --reserved-memory=""
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171703    2524 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171706    2524 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171709    2524 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171712    2524 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 04:06:11.178266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171715    2524 flags.go:64] FLAG: --runonce="false"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171718    2524 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171720    2524 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171723    2524 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171726    2524 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171730    2524 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171733    2524 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171736    2524 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171739    2524 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171742    2524 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171744    2524 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171747    2524 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171750    2524 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171753    2524 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171756    2524 flags.go:64] FLAG: --system-cgroups=""
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171759    2524 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171764    2524 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171766    2524 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171769    2524 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171773    2524 flags.go:64] FLAG: --tls-min-version=""
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171776    2524 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171779    2524 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171781    2524 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171784    2524 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171787    2524 flags.go:64] FLAG: --v="2"
Jul 15 04:06:11.178871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171791    2524 flags.go:64] FLAG: --version="false"
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171796    2524 flags.go:64] FLAG: --vmodule=""
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171800    2524 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.171803    2524 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171921    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171926    2524 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171929    2524 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171931    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171934    2524 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171937    2524 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171941    2524 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171943    2524 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171947    2524 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171950    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171953    2524 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171955    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171958    2524 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171960    2524 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171964    2524 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171966    2524 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:06:11.179495 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171970    2524 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171974    2524 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171977    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171980    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171982    2524 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171985    2524 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171988    2524 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171990    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171993    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171996    2524 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.171998    2524 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172001    2524 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172003    2524 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172006    2524 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172009    2524 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172011    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172015    2524 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172017    2524 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172020    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172022    2524 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:06:11.179987 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172025    2524 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172028    2524 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172032    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172034    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172039    2524 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172041    2524 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172045    2524 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172048    2524 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172051    2524 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172054    2524 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172057    2524 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172060    2524 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172063    2524 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172065    2524 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172068    2524 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172071    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172073    2524 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172076    2524 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172079    2524 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:06:11.180488 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172082    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172084    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172087    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172090    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172093    2524 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172095    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172099    2524 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172103    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172106    2524 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172108    2524 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172111    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172114    2524 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172116    2524 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172119    2524 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172121    2524 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172125    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172128    2524 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172131    2524 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172133    2524 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172136    2524 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:06:11.181058 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172138    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172141    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172143    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172146    2524 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172148    2524 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172151    2524 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172153    2524 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172156    2524 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172158    2524 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172161    2524 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172163    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172166    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172168    2524 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172170    2524 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172173    2524 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172175    2524 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172178    2524 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172180    2524 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172182    2524 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:06:11.181547 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172185    2524 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172187    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172190    2524 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172192    2524 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.172195    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.173083    2524 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.180458    2524 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.180473    2524 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.180495    2524 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.180501    2524 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180556    2524 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180561    2524 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180564    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180568    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180571    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:06:11.182012 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180574    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180577    2524 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180580    2524 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180583    2524 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180586    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180589    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180609    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180612    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180615    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180618    2524 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180621    2524 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180623    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180628    2524 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180631    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180634    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180637    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180640    2524 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180643    2524 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180645    2524 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:06:11.182374 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180647    2524 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180650    2524 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180653    2524 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180655    2524 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180658    2524 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180662    2524 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180664    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180667    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180669    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180672    2524 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180674    2524 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180678    2524 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180681    2524 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180684    2524 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180687    2524 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180690    2524 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180693    2524 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180696    2524 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180699    2524 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:06:11.182839 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180702    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180704    2524 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180707    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180710    2524 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180713    2524 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180716    2524 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180719    2524 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180722    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180725    2524 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180727    2524 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180730    2524 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180732    2524 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180735    2524 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180737    2524 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180740    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180742    2524 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180745    2524 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180747    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180750    2524 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180753    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:06:11.183288 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180756    2524 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180758    2524 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180761    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180763    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180767    2524 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180771    2524 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180773    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180776    2524 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180778    2524 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180781    2524 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180783    2524 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180786    2524 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180788    2524 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180791    2524 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180793    2524 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180796    2524 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180798    2524 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180800    2524 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180803    2524 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180805    2524 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:06:11.183761 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180808    2524 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180810    2524 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180813    2524 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180815    2524 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180819    2524 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180822    2524 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180825    2524 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180827    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180830    2524 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180833    2524 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180835    2524 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180838    2524 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180841    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180844    2524 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180846    2524 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:06:11.184237 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180849    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.180853    2524 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180967    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180971    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180975    2524 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180977    2524 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180980    2524 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180983    2524 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180985    2524 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180988    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180990    2524 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180992    2524 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180995    2524 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.180997    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181000    2524 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:06:11.184589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181002    2524 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181005    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181007    2524 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181010    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181012    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181015    2524 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181017    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181020    2524 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181022    2524 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181025    2524 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181027    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181030    2524 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181032    2524 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181035    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181037    2524 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181040    2524 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181043    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181045    2524 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181048    2524 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181050    2524 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:06:11.184989 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181053    2524 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181056    2524 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181058    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181060    2524 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181063    2524 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181066    2524 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181070    2524 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181074    2524 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181085    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181088    2524 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181090    2524 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181093    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181096    2524 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181098    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181101    2524 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181103    2524 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181106    2524 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181108    2524 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181111    2524 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:06:11.185478 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181113    2524 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181116    2524 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181118    2524 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181121    2524 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181124    2524 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181126    2524 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181128    2524 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181131    2524 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181133    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181138    2524 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181141    2524 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181144    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181147    2524 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181149    2524 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181152    2524 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181154    2524 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181157    2524 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181159    2524 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181162    2524 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:06:11.185935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181165    2524 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181167    2524 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181170    2524 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181172    2524 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181174    2524 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181177    2524 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181179    2524 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181182    2524 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181186    2524 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181189    2524 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181192    2524 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181195    2524 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181197    2524 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181200    2524 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181203    2524 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181206    2524 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181208    2524 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181211    2524 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181214    2524 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:06:11.186379 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181216    2524 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181219    2524 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181221    2524 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181224    2524 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181227    2524 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181229    2524 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181232    2524 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181234    2524 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:11.181237    2524 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.181242    2524 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.181350    2524 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.183235    2524 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.184698    2524 server.go:1019] "Starting client certificate rotation"
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.184794    2524 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 04:06:11.186965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.184834    2524 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 04:06:11.187324 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.187310    2524 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 04:06:11.189902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.189886    2524 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 04:06:11.203230 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.203206    2524 log.go:25] "Validated CRI v1 runtime API"
Jul 15 04:06:11.205156 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.205140    2524 log.go:25] "Validated CRI v1 image API"
Jul 15 04:06:11.205965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.205943    2524 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 04:06:11.211876 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.211857    2524 fs.go:135] Filesystem UUIDs: map[4f591bea-ab5e-435c-bd14-cb53d00e3f08:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9e4ab25c-23b4-4a48-a6d2-29cc5ec595cf:/dev/nvme0n1p3]
Jul 15 04:06:11.211936 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.211876    2524 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 04:06:11.214821 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.214800    2524 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 04:06:11.218209 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.218096    2524 manager.go:216] Machine: {Timestamp:2026-07-15 04:06:11.216473775 +0000 UTC m=+0.527855016 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098707 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f22a8780b911ef26bd323ab424fe5 SystemUUID:ec2f22a8-780b-911e-f26b-d323ab424fe5 BootID:e518b561-1b54-4830-80ae-d75af2918d3c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8a:9e:db:4a:27 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8a:9e:db:4a:27 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:1c:c4:f1:e9:0a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 04:06:11.218209 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.218204    2524 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 04:06:11.218302 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.218286    2524 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 04:06:11.219275 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.219252    2524 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 04:06:11.219419 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.219280    2524 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-76.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 04:06:11.219987 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.219976    2524 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 04:06:11.220020 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.219990    2524 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 04:06:11.220020 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.219998    2524 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 04:06:11.220887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.220876    2524 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 04:06:11.222150 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.222139    2524 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 04:06:11.225175 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.225162    2524 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 04:06:11.225304 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.225294    2524 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 04:06:11.225336 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.225329    2524 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 04:06:11.225364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.225344    2524 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 04:06:11.225398 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.225369    2524 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 04:06:11.225398 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.225386    2524 kubelet.go:398] "Adding apiserver pod source"
Jul 15 04:06:11.225455 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.225400    2524 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 04:06:11.225738 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.225727    2524 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 04:06:11.225773 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.225748    2524 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 04:06:11.227910 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.227888    2524 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 04:06:11.230365 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.230350    2524 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 04:06:11.230408 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.230383    2524 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 04:06:11.230551 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.230537    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 04:06:11.230631 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.230554    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 04:06:11.230631 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.230560    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 04:06:11.230631 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.230566    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 04:06:11.230631 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.230575    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 04:06:11.230631 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.230583    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 04:06:11.231234 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.231224    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 04:06:11.231234 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.231234    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 04:06:11.231348 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.231276    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 04:06:11.231348 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.231283    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 04:06:11.231348 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.231296    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 04:06:11.231348 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.231305    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 04:06:11.231348 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.231327    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 04:06:11.231348 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.231333    2524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 04:06:11.234601 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.234587    2524 server.go:1268] "Started kubelet"
Jul 15 04:06:11.234829 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.234773    2524 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 04:06:11.234829 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.234817    2524 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bzq8t"
Jul 15 04:06:11.235487 ip-10-0-129-76 systemd[1]: Started Kubernetes Kubelet.
Jul 15 04:06:11.241387 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.234748    2524 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 04:06:11.241543 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.241514    2524 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 04:06:11.241896 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.241869    2524 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bzq8t"
Jul 15 04:06:11.242961 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.242936    2524 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 04:06:11.243141 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.243120    2524 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-76.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 04:06:11.243288 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:11.243253    2524 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 04:06:11.243419 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.243404    2524 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 04:06:11.243735 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:11.243710    2524 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 04:06:11.248987 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.248967    2524 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 04:06:11.248987 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.248979    2524 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 04:06:11.249444 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:11.249118    2524 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-76.ec2.internal\" not found"
Jul 15 04:06:11.249991 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.249969    2524 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 04:06:11.249991 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.249990    2524 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 04:06:11.250147 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.250106    2524 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 04:06:11.250147 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.250115    2524 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 04:06:11.250147 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.250143    2524 factory.go:55] Registering systemd factory
Jul 15 04:06:11.250272 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.250185    2524 factory.go:223] Registration of the systemd container factory successfully
Jul 15 04:06:11.250428 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.250404    2524 factory.go:153] Registering CRI-O factory
Jul 15 04:06:11.250428 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.250423    2524 factory.go:223] Registration of the crio container factory successfully
Jul 15 04:06:11.250626 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.250478    2524 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 04:06:11.250626 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.250503    2524 factory.go:103] Registering Raw factory
Jul 15 04:06:11.250626 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.250519    2524 manager.go:1196] Started watching for new ooms in manager
Jul 15 04:06:11.250626 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.250587    2524 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 04:06:11.251194 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.251178    2524 manager.go:349] Starting recovery of all containers
Jul 15 04:06:11.252605 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.252581    2524 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:06:11.254464 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.254010    2524 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:06:11.256964 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:11.256932    2524 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-76.ec2.internal\" not found" node="ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.261641 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.261463    2524 manager.go:354] Recovery completed
Jul 15 04:06:11.265833 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.265819    2524 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:06:11.267983 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.267964    2524 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-76.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:06:11.268037 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.268003    2524 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-76.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:06:11.268037 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.268020    2524 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-76.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:06:11.268523 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.268511    2524 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 04:06:11.268582 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.268523    2524 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 04:06:11.268582 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.268539    2524 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 04:06:11.268582 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.268577    2524 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:06:11.269733 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.269721    2524 policy_none.go:49] "None policy: Start"
Jul 15 04:06:11.269768 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.269737    2524 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 04:06:11.269768 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.269748    2524 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 04:06:11.269914 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.269897    2524 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-76.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:06:11.269965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.269934    2524 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-76.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:06:11.269965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.269950    2524 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-76.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:06:11.271003 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.270985    2524 policy_none.go:47] "Start"
Jul 15 04:06:11.310336 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.310320    2524 manager.go:337] "Starting Device Plugin manager"
Jul 15 04:06:11.327325 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:11.310352    2524 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 04:06:11.327325 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.310364    2524 server.go:87] "Starting device plugin registration server"
Jul 15 04:06:11.327325 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.310598    2524 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 04:06:11.327325 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.310613    2524 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 04:06:11.327325 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.310700    2524 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 04:06:11.327325 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.310830    2524 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 04:06:11.327325 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.310842    2524 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 04:06:11.327325 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:11.311298    2524 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 04:06:11.327325 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:11.311355    2524 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-76.ec2.internal\" not found"
Jul 15 04:06:11.411509 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.411482    2524 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:06:11.412554 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.412535    2524 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-76.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:06:11.412658 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.412568    2524 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-76.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:06:11.412658 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.412579    2524 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-76.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:06:11.412658 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.412630    2524 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.421997 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.421973    2524 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.424386 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.424364    2524 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 04:06:11.425821 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.425797    2524 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 04:06:11.425910 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.425854    2524 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 04:06:11.425910 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.425880    2524 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 04:06:11.426008 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:11.425923    2524 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 04:06:11.428197 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.428178    2524 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:06:11.531649 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.531528    2524 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-76.ec2.internal"]
Jul 15 04:06:11.534245 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.534225    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.536286 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.536269    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.562365 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.562337    2524 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.566924 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.566909    2524 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.581755 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.581735    2524 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 04:06:11.595413 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.595395    2524 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 04:06:11.652637 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.652570    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6faa769590e33e316d4b91d47d08166c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal\" (UID: \"6faa769590e33e316d4b91d47d08166c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.652713 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.652640    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6faa769590e33e316d4b91d47d08166c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal\" (UID: \"6faa769590e33e316d4b91d47d08166c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.652713 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.652662    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f15257a2672d183728dc10e769d94d89-config\") pod \"kube-apiserver-proxy-ip-10-0-129-76.ec2.internal\" (UID: \"f15257a2672d183728dc10e769d94d89\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.753537 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.753508    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6faa769590e33e316d4b91d47d08166c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal\" (UID: \"6faa769590e33e316d4b91d47d08166c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.753537 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.753538    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f15257a2672d183728dc10e769d94d89-config\") pod \"kube-apiserver-proxy-ip-10-0-129-76.ec2.internal\" (UID: \"f15257a2672d183728dc10e769d94d89\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.753730 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.753556    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6faa769590e33e316d4b91d47d08166c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal\" (UID: \"6faa769590e33e316d4b91d47d08166c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.753730 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.753598    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6faa769590e33e316d4b91d47d08166c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal\" (UID: \"6faa769590e33e316d4b91d47d08166c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.753730 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.753653    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f15257a2672d183728dc10e769d94d89-config\") pod \"kube-apiserver-proxy-ip-10-0-129-76.ec2.internal\" (UID: \"f15257a2672d183728dc10e769d94d89\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.753730 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.753655    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6faa769590e33e316d4b91d47d08166c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal\" (UID: \"6faa769590e33e316d4b91d47d08166c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.886000 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.885913    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:11.898814 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:11.898790    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal"
Jul 15 04:06:12.185021 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.184941    2524 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 04:06:12.185565 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.185077    2524 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 04:06:12.185565 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.185126    2524 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 04:06:12.226166 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.226142    2524 apiserver.go:52] "Watching apiserver"
Jul 15 04:06:12.235119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.235090    2524 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 04:06:12.237325 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.237291    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-d9v7v","kube-system/kube-apiserver-proxy-ip-10-0-129-76.ec2.internal","openshift-image-registry/node-ca-9f68m","openshift-multus/multus-zcjw5","openshift-multus/network-metrics-daemon-86pgb","openshift-network-diagnostics/network-check-target-txk5x","openshift-network-operator/iptables-alerter-8hzpl","openshift-ovn-kubernetes/ovnkube-node-kmwrc","kube-system/global-pull-secret-syncer-swf7w","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg","openshift-cluster-node-tuning-operator/tuned-ntm4b","openshift-dns/node-resolver-kjdkn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal","openshift-multus/multus-additional-cni-plugins-vrr2l"]
Jul 15 04:06:12.240205 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.240183    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.242540 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.242521    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:12.242664 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.242637    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 04:06:12.242777 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.242753    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cz6wv\""
Jul 15 04:06:12.242920 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.242822    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 04:06:12.243090 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.243078    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 04:06:12.244893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.244875    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.245069 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.245049    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 04:06:12.245304 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.245285    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bj2d7\""
Jul 15 04:06:12.245388 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.245289    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 04:06:12.247092 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.247073    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 04:06:12.247159 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.247148    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.247218 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.247184    2524 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 04:01:11 +0000 UTC" deadline="2028-04-10 07:01:58.872215204 +0000 UTC"
Jul 15 04:06:12.247259 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.247218    2524 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15242h55m46.624999731s"
Jul 15 04:06:12.247426 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.247405    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 04:06:12.247426 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.247427    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rlgld\""
Jul 15 04:06:12.247590 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.247573    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 04:06:12.247859 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.247846    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 04:06:12.248123 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.248110    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 04:06:12.249256 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.249241    2524 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 04:06:12.249460 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.249444    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 04:06:12.249967 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.249680    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.249967 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.249762    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mw75n\""
Jul 15 04:06:12.252174 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.252156    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:12.252339 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.252310    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:12.252504 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.252489    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 04:06:12.253598 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.253578    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 04:06:12.253716 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.253579    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 04:06:12.253772 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.253580    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c5xv5\""
Jul 15 04:06:12.253772 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.253579    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 04:06:12.253846 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.253581    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 04:06:12.253846 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.253586    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 04:06:12.254608 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.254571    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kjdkn"
Jul 15 04:06:12.256902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.256885    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 04:06:12.257056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.257038    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 04:06:12.257177 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.257153    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.257273 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.257257    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vh6qr\""
Jul 15 04:06:12.259274 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259246    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-cnibin\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.259372 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259284    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-var-lib-cni-bin\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.259372 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259303    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-var-lib-kubelet\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.259372 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259344    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-etc-openvswitch\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.259538 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259383    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f853b65c-ad5d-4187-b60b-bdb5715a1873-ovnkube-config\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.259538 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259427    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/04752185-efbf-4166-8b68-7834e27cb871-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.259538 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259457    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-device-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.259538 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259484    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-etc-selinux\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.259538 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259509    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-system-cni-dir\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.259538 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259516    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pmzrf\""
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259540    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-var-lib-cni-multus\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259570    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-slash\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259585    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259595    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-node-log\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259658    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-conf-dir\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259684    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-run-multus-certs\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259709    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4tx4\" (UniqueName: \"kubernetes.io/projected/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-kube-api-access-v4tx4\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259544    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259733    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-cni-binary-copy\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259772    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-socket-dir-parent\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259803    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259804    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-cnibin\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.259803    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:12.259851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259833    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89n4k\" (UniqueName: \"kubernetes.io/projected/04752185-efbf-4166-8b68-7834e27cb871-kube-api-access-89n4k\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259881    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-log-socket\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259906    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-registration-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259930    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-run-k8s-cni-cncf-io\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259955    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-systemd-units\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.259980    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f853b65c-ad5d-4187-b60b-bdb5715a1873-ovn-node-metrics-cert\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260018    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-system-cni-dir\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260042    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-run-ovn\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260073    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-cni-netd\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260101    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260119    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58f571a6-f29b-4885-bac5-8e637b7a2027-agent-certs\") pod \"konnectivity-agent-d9v7v\" (UID: \"58f571a6-f29b-4885-bac5-8e637b7a2027\") " pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260137    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/04752185-efbf-4166-8b68-7834e27cb871-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260162    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-etc-kubernetes\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260215    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f853b65c-ad5d-4187-b60b-bdb5715a1873-env-overrides\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260246    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58f571a6-f29b-4885-bac5-8e637b7a2027-konnectivity-ca\") pod \"konnectivity-agent-d9v7v\" (UID: \"58f571a6-f29b-4885-bac5-8e637b7a2027\") " pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260267    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-os-release\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.260440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260285    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsfmr\" (UniqueName: \"kubernetes.io/projected/f853b65c-ad5d-4187-b60b-bdb5715a1873-kube-api-access-xsfmr\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260308    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260331    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-socket-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260348    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-cni-bin\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260386    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-run-ovn-kubernetes\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260425    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f853b65c-ad5d-4187-b60b-bdb5715a1873-ovnkube-script-lib\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260442    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-daemon-config\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260482    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-run-systemd\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260503    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-run-openvswitch\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260543    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dngwl\" (UniqueName: \"kubernetes.io/projected/1e94d729-dba7-454d-863f-2b9c091e65af-kube-api-access-dngwl\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260570    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-os-release\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260597    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04752185-efbf-4166-8b68-7834e27cb871-cni-binary-copy\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260659    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-kubelet\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.261056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260758    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-run-netns\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.263480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.260796    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-hostroot\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.263480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.263045    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.263480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.263087    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twpb4\" (UniqueName: \"kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4\") pod \"network-check-target-txk5x\" (UID: \"5dcea033-1394-4680-b043-b24b294ccd7a\") " pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:12.263480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.263125    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-cni-dir\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.263480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.263156    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-run-netns\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.263480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.263182    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-var-lib-openvswitch\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.263480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.263216    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-sys-fs\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.263892 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.263615    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:12.263892 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.263714    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:12.265430 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.265406    2524 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 04:06:12.266345 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.266328    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8hzpl"
Jul 15 04:06:12.268559 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.268539    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 04:06:12.268559 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.268549    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 04:06:12.268877 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.268864    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 04:06:12.269351 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.269332    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9f68m"
Jul 15 04:06:12.269596 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.269582    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7klc2\""
Jul 15 04:06:12.271410 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.271390    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 04:06:12.271596 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.271580    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 04:06:12.271803 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.271788    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xj2s8\""
Jul 15 04:06:12.271883 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.271792    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 04:06:12.302921 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.302900    2524 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4c8kn"
Jul 15 04:06:12.310534 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.310519    2524 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4c8kn"
Jul 15 04:06:12.351489 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.351469    2524 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 04:06:12.363160 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363137    2524 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:06:12.363393 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363375    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58f571a6-f29b-4885-bac5-8e637b7a2027-konnectivity-ca\") pod \"konnectivity-agent-d9v7v\" (UID: \"58f571a6-f29b-4885-bac5-8e637b7a2027\") " pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:12.363454 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363404    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-sysctl-conf\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.363454 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363423    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-systemd\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.363454 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363440    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2thxx\" (UniqueName: \"kubernetes.io/projected/afb4f7f2-d8c9-4451-ae39-97507126f420-kube-api-access-2thxx\") pod \"node-ca-9f68m\" (UID: \"afb4f7f2-d8c9-4451-ae39-97507126f420\") " pod="openshift-image-registry/node-ca-9f68m"
Jul 15 04:06:12.363572 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363468    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-os-release\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.363572 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363531    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsfmr\" (UniqueName: \"kubernetes.io/projected/f853b65c-ad5d-4187-b60b-bdb5715a1873-kube-api-access-xsfmr\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.363695 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363567    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.363695 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363569    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-os-release\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.363695 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363593    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-socket-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.363695 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363618    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-cni-bin\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.363695 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363635    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.363695 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363650    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-sysconfig\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.363695 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363693    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-sys\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.364054 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363701    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-cni-bin\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.364054 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363719    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:12.364054 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363758    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-run-ovn-kubernetes\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.364054 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363774    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f853b65c-ad5d-4187-b60b-bdb5715a1873-ovnkube-script-lib\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.364054 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363903    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtb4w\" (UniqueName: \"kubernetes.io/projected/77b773b0-2e42-47c2-ab57-92a0162cf8db-kube-api-access-xtb4w\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.364054 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363947    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-kubelet-config\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:12.364054 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.363989    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-daemon-config\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.364054 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364024    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-run-systemd\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364080    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-run-systemd\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364150    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-run-openvswitch\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364175    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dngwl\" (UniqueName: \"kubernetes.io/projected/1e94d729-dba7-454d-863f-2b9c091e65af-kube-api-access-dngwl\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364194    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-os-release\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364210    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04752185-efbf-4166-8b68-7834e27cb871-cni-binary-copy\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364216    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-run-openvswitch\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364227    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-kubelet\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364246    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-modprobe-d\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364245    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-socket-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364265    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-dbus\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364289    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-os-release\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364293    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-run-ovn-kubernetes\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364338    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-kubelet\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.364457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364457    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-run-netns\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364488    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-hostroot\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364536    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-run-netns\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364539    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-hostroot\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364580    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364605    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58f571a6-f29b-4885-bac5-8e637b7a2027-konnectivity-ca\") pod \"konnectivity-agent-d9v7v\" (UID: \"58f571a6-f29b-4885-bac5-8e637b7a2027\") " pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364614    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f853b65c-ad5d-4187-b60b-bdb5715a1873-ovnkube-script-lib\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364631    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-daemon-config\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364630    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpnrx\" (UniqueName: \"kubernetes.io/projected/3d7355a2-bc8b-435e-8c31-60297ecc4e35-kube-api-access-dpnrx\") pod \"node-resolver-kjdkn\" (UID: \"3d7355a2-bc8b-435e-8c31-60297ecc4e35\") " pod="openshift-dns/node-resolver-kjdkn"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364681    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-run\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364705    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afb4f7f2-d8c9-4451-ae39-97507126f420-host\") pod \"node-ca-9f68m\" (UID: \"afb4f7f2-d8c9-4451-ae39-97507126f420\") " pod="openshift-image-registry/node-ca-9f68m"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364743    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twpb4\" (UniqueName: \"kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4\") pod \"network-check-target-txk5x\" (UID: \"5dcea033-1394-4680-b043-b24b294ccd7a\") " pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364752    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364759    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3d7355a2-bc8b-435e-8c31-60297ecc4e35-hosts-file\") pod \"node-resolver-kjdkn\" (UID: \"3d7355a2-bc8b-435e-8c31-60297ecc4e35\") " pod="openshift-dns/node-resolver-kjdkn"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364809    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d7355a2-bc8b-435e-8c31-60297ecc4e35-tmp-dir\") pod \"node-resolver-kjdkn\" (UID: \"3d7355a2-bc8b-435e-8c31-60297ecc4e35\") " pod="openshift-dns/node-resolver-kjdkn"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364835    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04752185-efbf-4166-8b68-7834e27cb871-cni-binary-copy\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364838    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-cni-dir\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364897    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-run-netns\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364924    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-var-lib-openvswitch\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364947    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-run-netns\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364961    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-cni-dir\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364963    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-sys-fs\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364988    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-cnibin\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.364996    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-var-lib-openvswitch\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365014    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-var-lib-cni-bin\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365037    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-sys-fs\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365047    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-cnibin\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365039    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-var-lib-kubelet\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365079    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-var-lib-kubelet\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365062    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-var-lib-cni-bin\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365089    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-etc-openvswitch\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365117    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f853b65c-ad5d-4187-b60b-bdb5715a1873-ovnkube-config\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365141    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/04752185-efbf-4166-8b68-7834e27cb871-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365156    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-etc-openvswitch\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365171    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-sysctl-d\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.365893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365196    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77b773b0-2e42-47c2-ab57-92a0162cf8db-tmp\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365229    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-device-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365254    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-etc-selinux\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365281    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-system-cni-dir\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365291    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-device-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365311    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-var-lib-cni-multus\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365360    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-system-cni-dir\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365364    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-slash\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365400    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-etc-selinux\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365409    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-slash\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365416    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-var-lib-cni-multus\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365423    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-node-log\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365446    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-conf-dir\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365463    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-run-multus-certs\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365483    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4tx4\" (UniqueName: \"kubernetes.io/projected/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-kube-api-access-v4tx4\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365512    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jngg\" (UniqueName: \"kubernetes.io/projected/01875d97-1469-4a47-a2df-2e43893ec711-kube-api-access-7jngg\") pod \"iptables-alerter-8hzpl\" (UID: \"01875d97-1469-4a47-a2df-2e43893ec711\") " pod="openshift-network-operator/iptables-alerter-8hzpl"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365525    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-node-log\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.366466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365528    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-run-multus-certs\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365577    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f853b65c-ad5d-4187-b60b-bdb5715a1873-ovnkube-config\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365603    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/04752185-efbf-4166-8b68-7834e27cb871-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365678    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afb4f7f2-d8c9-4451-ae39-97507126f420-serviceca\") pod \"node-ca-9f68m\" (UID: \"afb4f7f2-d8c9-4451-ae39-97507126f420\") " pod="openshift-image-registry/node-ca-9f68m"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365710    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-cni-binary-copy\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365751    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-socket-dir-parent\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365771    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-conf-dir\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365777    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-cnibin\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365805    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89n4k\" (UniqueName: \"kubernetes.io/projected/04752185-efbf-4166-8b68-7834e27cb871-kube-api-access-89n4k\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365829    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-lib-modules\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365853    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365877    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01875d97-1469-4a47-a2df-2e43893ec711-host-slash\") pod \"iptables-alerter-8hzpl\" (UID: \"01875d97-1469-4a47-a2df-2e43893ec711\") " pod="openshift-network-operator/iptables-alerter-8hzpl"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365901    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-log-socket\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365925    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-registration-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365949    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-host\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365972    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/01875d97-1469-4a47-a2df-2e43893ec711-iptables-alerter-script\") pod \"iptables-alerter-8hzpl\" (UID: \"01875d97-1469-4a47-a2df-2e43893ec711\") " pod="openshift-network-operator/iptables-alerter-8hzpl"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.365999    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-run-k8s-cni-cncf-io\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.367030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366024    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-systemd-units\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366049    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f853b65c-ad5d-4187-b60b-bdb5715a1873-ovn-node-metrics-cert\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366118    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-multus-socket-dir-parent\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366155    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-cni-binary-copy\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366177    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-log-socket\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366180    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04752185-efbf-4166-8b68-7834e27cb871-cnibin\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366120    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-var-lib-kubelet\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366248    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-tuned\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366254    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e94d729-dba7-454d-863f-2b9c091e65af-registration-dir\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366282    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-system-cni-dir\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366317    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-run-ovn\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366321    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-host-run-k8s-cni-cncf-io\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366342    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-cni-netd\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366416    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-systemd-units\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366424    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366448    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-system-cni-dir\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366458    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58f571a6-f29b-4885-bac5-8e637b7a2027-agent-certs\") pod \"konnectivity-agent-d9v7v\" (UID: \"58f571a6-f29b-4885-bac5-8e637b7a2027\") " pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:12.367874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366484    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-run-ovn\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.368766 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366484    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-cni-netd\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.368766 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366486    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/04752185-efbf-4166-8b68-7834e27cb871-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.368766 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366535    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f853b65c-ad5d-4187-b60b-bdb5715a1873-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.368766 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366569    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-kubernetes\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.368766 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366600    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzs86\" (UniqueName: \"kubernetes.io/projected/d62c63a3-e8fd-424a-811b-4e65e545f33a-kube-api-access-gzs86\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:12.368766 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366654    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-etc-kubernetes\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.368766 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366681    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f853b65c-ad5d-4187-b60b-bdb5715a1873-env-overrides\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.368766 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.366805    2524 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 04:06:12.368766 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.367026    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/04752185-efbf-4166-8b68-7834e27cb871-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.368766 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.367191    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-etc-kubernetes\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.368766 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.367471    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f853b65c-ad5d-4187-b60b-bdb5715a1873-env-overrides\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.373911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.371006    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58f571a6-f29b-4885-bac5-8e637b7a2027-agent-certs\") pod \"konnectivity-agent-d9v7v\" (UID: \"58f571a6-f29b-4885-bac5-8e637b7a2027\") " pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:12.373911 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.371379    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:06:12.373911 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.371413    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:06:12.373911 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.371430    2524 projected.go:196] Error preparing data for projected volume kube-api-access-twpb4 for pod openshift-network-diagnostics/network-check-target-txk5x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:12.373911 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.371520    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4 podName:5dcea033-1394-4680-b043-b24b294ccd7a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:12.871496027 +0000 UTC m=+2.182877282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-twpb4" (UniqueName: "kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4") pod "network-check-target-txk5x" (UID: "5dcea033-1394-4680-b043-b24b294ccd7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:12.373911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.373306    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsfmr\" (UniqueName: \"kubernetes.io/projected/f853b65c-ad5d-4187-b60b-bdb5715a1873-kube-api-access-xsfmr\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.373911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.373838    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f853b65c-ad5d-4187-b60b-bdb5715a1873-ovn-node-metrics-cert\") pod \"ovnkube-node-kmwrc\" (UID: \"f853b65c-ad5d-4187-b60b-bdb5715a1873\") " pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.373911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.373841    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dngwl\" (UniqueName: \"kubernetes.io/projected/1e94d729-dba7-454d-863f-2b9c091e65af-kube-api-access-dngwl\") pod \"aws-ebs-csi-driver-node-pkszg\" (UID: \"1e94d729-dba7-454d-863f-2b9c091e65af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.374405 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.374046    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4tx4\" (UniqueName: \"kubernetes.io/projected/abfb7bf3-6f37-4fcd-9909-2483c2f5315f-kube-api-access-v4tx4\") pod \"multus-zcjw5\" (UID: \"abfb7bf3-6f37-4fcd-9909-2483c2f5315f\") " pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.377214 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.377192    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89n4k\" (UniqueName: \"kubernetes.io/projected/04752185-efbf-4166-8b68-7834e27cb871-kube-api-access-89n4k\") pod \"multus-additional-cni-plugins-vrr2l\" (UID: \"04752185-efbf-4166-8b68-7834e27cb871\") " pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.467474 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467391    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpnrx\" (UniqueName: \"kubernetes.io/projected/3d7355a2-bc8b-435e-8c31-60297ecc4e35-kube-api-access-dpnrx\") pod \"node-resolver-kjdkn\" (UID: \"3d7355a2-bc8b-435e-8c31-60297ecc4e35\") " pod="openshift-dns/node-resolver-kjdkn"
Jul 15 04:06:12.467474 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467425    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-run\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.467474 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467448    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afb4f7f2-d8c9-4451-ae39-97507126f420-host\") pod \"node-ca-9f68m\" (UID: \"afb4f7f2-d8c9-4451-ae39-97507126f420\") " pod="openshift-image-registry/node-ca-9f68m"
Jul 15 04:06:12.467878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467495    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afb4f7f2-d8c9-4451-ae39-97507126f420-host\") pod \"node-ca-9f68m\" (UID: \"afb4f7f2-d8c9-4451-ae39-97507126f420\") " pod="openshift-image-registry/node-ca-9f68m"
Jul 15 04:06:12.467878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467571    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3d7355a2-bc8b-435e-8c31-60297ecc4e35-hosts-file\") pod \"node-resolver-kjdkn\" (UID: \"3d7355a2-bc8b-435e-8c31-60297ecc4e35\") " pod="openshift-dns/node-resolver-kjdkn"
Jul 15 04:06:12.467878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467632    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d7355a2-bc8b-435e-8c31-60297ecc4e35-tmp-dir\") pod \"node-resolver-kjdkn\" (UID: \"3d7355a2-bc8b-435e-8c31-60297ecc4e35\") " pod="openshift-dns/node-resolver-kjdkn"
Jul 15 04:06:12.467878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467575    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-run\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.467878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467638    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3d7355a2-bc8b-435e-8c31-60297ecc4e35-hosts-file\") pod \"node-resolver-kjdkn\" (UID: \"3d7355a2-bc8b-435e-8c31-60297ecc4e35\") " pod="openshift-dns/node-resolver-kjdkn"
Jul 15 04:06:12.467878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467669    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-sysctl-d\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.467878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467762    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77b773b0-2e42-47c2-ab57-92a0162cf8db-tmp\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.467878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467803    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jngg\" (UniqueName: \"kubernetes.io/projected/01875d97-1469-4a47-a2df-2e43893ec711-kube-api-access-7jngg\") pod \"iptables-alerter-8hzpl\" (UID: \"01875d97-1469-4a47-a2df-2e43893ec711\") " pod="openshift-network-operator/iptables-alerter-8hzpl"
Jul 15 04:06:12.467878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467830    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afb4f7f2-d8c9-4451-ae39-97507126f420-serviceca\") pod \"node-ca-9f68m\" (UID: \"afb4f7f2-d8c9-4451-ae39-97507126f420\") " pod="openshift-image-registry/node-ca-9f68m"
Jul 15 04:06:12.467878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467862    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-lib-modules\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.467878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467803    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-sysctl-d\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467888    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467913    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01875d97-1469-4a47-a2df-2e43893ec711-host-slash\") pod \"iptables-alerter-8hzpl\" (UID: \"01875d97-1469-4a47-a2df-2e43893ec711\") " pod="openshift-network-operator/iptables-alerter-8hzpl"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467969    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01875d97-1469-4a47-a2df-2e43893ec711-host-slash\") pod \"iptables-alerter-8hzpl\" (UID: \"01875d97-1469-4a47-a2df-2e43893ec711\") " pod="openshift-network-operator/iptables-alerter-8hzpl"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.467989    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d7355a2-bc8b-435e-8c31-60297ecc4e35-tmp-dir\") pod \"node-resolver-kjdkn\" (UID: \"3d7355a2-bc8b-435e-8c31-60297ecc4e35\") " pod="openshift-dns/node-resolver-kjdkn"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.468026    2524 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468032    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-lib-modules\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.468100    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret podName:bb2bdcea-aedf-4f70-8aae-b2901b9c4a83 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:12.968085149 +0000 UTC m=+2.279466394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret") pod "global-pull-secret-syncer-swf7w" (UID: "bb2bdcea-aedf-4f70-8aae-b2901b9c4a83") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468145    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-host\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468188    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/01875d97-1469-4a47-a2df-2e43893ec711-iptables-alerter-script\") pod \"iptables-alerter-8hzpl\" (UID: \"01875d97-1469-4a47-a2df-2e43893ec711\") " pod="openshift-network-operator/iptables-alerter-8hzpl"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468217    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-var-lib-kubelet\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468230    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-host\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468242    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-tuned\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468268    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-kubernetes\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468283    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzs86\" (UniqueName: \"kubernetes.io/projected/d62c63a3-e8fd-424a-811b-4e65e545f33a-kube-api-access-gzs86\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468285    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-var-lib-kubelet\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468301    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-sysctl-conf\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.468467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468331    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-systemd\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468359    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2thxx\" (UniqueName: \"kubernetes.io/projected/afb4f7f2-d8c9-4451-ae39-97507126f420-kube-api-access-2thxx\") pod \"node-ca-9f68m\" (UID: \"afb4f7f2-d8c9-4451-ae39-97507126f420\") " pod="openshift-image-registry/node-ca-9f68m"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468363    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-kubernetes\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468392    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-sysconfig\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468405    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-sysctl-conf\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468419    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-sys\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468509    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afb4f7f2-d8c9-4451-ae39-97507126f420-serviceca\") pod \"node-ca-9f68m\" (UID: \"afb4f7f2-d8c9-4451-ae39-97507126f420\") " pod="openshift-image-registry/node-ca-9f68m"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468568    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468576    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-sysconfig\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468600    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtb4w\" (UniqueName: \"kubernetes.io/projected/77b773b0-2e42-47c2-ab57-92a0162cf8db-kube-api-access-xtb4w\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468630    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-kubelet-config\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468677    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-sys\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468686    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-modprobe-d\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468713    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-dbus\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.468722    2524 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468762    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-kubelet-config\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468722    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-systemd\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.469657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468798    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-dbus\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:12.470576 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.468805    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs podName:d62c63a3-e8fd-424a-811b-4e65e545f33a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:12.968767406 +0000 UTC m=+2.280148636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs") pod "network-metrics-daemon-86pgb" (UID: "d62c63a3-e8fd-424a-811b-4e65e545f33a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:12.470576 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468814    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/01875d97-1469-4a47-a2df-2e43893ec711-iptables-alerter-script\") pod \"iptables-alerter-8hzpl\" (UID: \"01875d97-1469-4a47-a2df-2e43893ec711\") " pod="openshift-network-operator/iptables-alerter-8hzpl"
Jul 15 04:06:12.470576 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.468870    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-modprobe-d\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.470718 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.470688    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/77b773b0-2e42-47c2-ab57-92a0162cf8db-etc-tuned\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.470755 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.470746    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77b773b0-2e42-47c2-ab57-92a0162cf8db-tmp\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.487621 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.487583    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2thxx\" (UniqueName: \"kubernetes.io/projected/afb4f7f2-d8c9-4451-ae39-97507126f420-kube-api-access-2thxx\") pod \"node-ca-9f68m\" (UID: \"afb4f7f2-d8c9-4451-ae39-97507126f420\") " pod="openshift-image-registry/node-ca-9f68m"
Jul 15 04:06:12.487996 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.487976    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzs86\" (UniqueName: \"kubernetes.io/projected/d62c63a3-e8fd-424a-811b-4e65e545f33a-kube-api-access-gzs86\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:12.488093 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.488068    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpnrx\" (UniqueName: \"kubernetes.io/projected/3d7355a2-bc8b-435e-8c31-60297ecc4e35-kube-api-access-dpnrx\") pod \"node-resolver-kjdkn\" (UID: \"3d7355a2-bc8b-435e-8c31-60297ecc4e35\") " pod="openshift-dns/node-resolver-kjdkn"
Jul 15 04:06:12.488359 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.488331    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jngg\" (UniqueName: \"kubernetes.io/projected/01875d97-1469-4a47-a2df-2e43893ec711-kube-api-access-7jngg\") pod \"iptables-alerter-8hzpl\" (UID: \"01875d97-1469-4a47-a2df-2e43893ec711\") " pod="openshift-network-operator/iptables-alerter-8hzpl"
Jul 15 04:06:12.488448 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.488404    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtb4w\" (UniqueName: \"kubernetes.io/projected/77b773b0-2e42-47c2-ab57-92a0162cf8db-kube-api-access-xtb4w\") pod \"tuned-ntm4b\" (UID: \"77b773b0-2e42-47c2-ab57-92a0162cf8db\") " pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.561057 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.561026    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg"
Jul 15 04:06:12.579405 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.579372    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:12.598480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.598454    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vrr2l"
Jul 15 04:06:12.616217 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:12.616188    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04752185_efbf_4166_8b68_7834e27cb871.slice/crio-8cd02eaf1acf2ed981dc67b70232bd1e94990f42875845b3d631f6f70bc723a5 WatchSource:0}: Error finding container 8cd02eaf1acf2ed981dc67b70232bd1e94990f42875845b3d631f6f70bc723a5: Status 404 returned error can't find the container with id 8cd02eaf1acf2ed981dc67b70232bd1e94990f42875845b3d631f6f70bc723a5
Jul 15 04:06:12.616997 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:12.616976    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6faa769590e33e316d4b91d47d08166c.slice/crio-c8b9a5354b936937c3f4ba6fa3a63a15b789bd95be2cdf66bbc3a1d71eec82ae WatchSource:0}: Error finding container c8b9a5354b936937c3f4ba6fa3a63a15b789bd95be2cdf66bbc3a1d71eec82ae: Status 404 returned error can't find the container with id c8b9a5354b936937c3f4ba6fa3a63a15b789bd95be2cdf66bbc3a1d71eec82ae
Jul 15 04:06:12.617580 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:12.617558    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf15257a2672d183728dc10e769d94d89.slice/crio-0a501846a4893731151db05f8059a0b9407d55713285ca89a8944df5c6011753 WatchSource:0}: Error finding container 0a501846a4893731151db05f8059a0b9407d55713285ca89a8944df5c6011753: Status 404 returned error can't find the container with id 0a501846a4893731151db05f8059a0b9407d55713285ca89a8944df5c6011753
Jul 15 04:06:12.618584 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:12.618564    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58f571a6_f29b_4885_bac5_8e637b7a2027.slice/crio-ad8d1ec3bc3c0dbe522c544d6e78ce5902040400844c695f4ea1423cc506cba1 WatchSource:0}: Error finding container ad8d1ec3bc3c0dbe522c544d6e78ce5902040400844c695f4ea1423cc506cba1: Status 404 returned error can't find the container with id ad8d1ec3bc3c0dbe522c544d6e78ce5902040400844c695f4ea1423cc506cba1
Jul 15 04:06:12.621412 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.621397    2524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 04:06:12.623120 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.623102    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zcjw5"
Jul 15 04:06:12.634107 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:12.634086    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabfb7bf3_6f37_4fcd_9909_2483c2f5315f.slice/crio-2fa251dbef65f652429d22cbd479a0dd8b599d939f1be1848494c8b7d932644e WatchSource:0}: Error finding container 2fa251dbef65f652429d22cbd479a0dd8b599d939f1be1848494c8b7d932644e: Status 404 returned error can't find the container with id 2fa251dbef65f652429d22cbd479a0dd8b599d939f1be1848494c8b7d932644e
Jul 15 04:06:12.641411 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.641392    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:12.644194 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.644176    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kjdkn"
Jul 15 04:06:12.650357 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:12.650329    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf853b65c_ad5d_4187_b60b_bdb5715a1873.slice/crio-12b696b4cff5c015f3c685ff3511f32ed7f7d7394c273f2c4666ca9dabc64e40 WatchSource:0}: Error finding container 12b696b4cff5c015f3c685ff3511f32ed7f7d7394c273f2c4666ca9dabc64e40: Status 404 returned error can't find the container with id 12b696b4cff5c015f3c685ff3511f32ed7f7d7394c273f2c4666ca9dabc64e40
Jul 15 04:06:12.653370 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:12.653350    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d7355a2_bc8b_435e_8c31_60297ecc4e35.slice/crio-caabbbe25d6163c66a82e5346fa4461a7f4b2448e3fbbfcd26003aebc5565ed5 WatchSource:0}: Error finding container caabbbe25d6163c66a82e5346fa4461a7f4b2448e3fbbfcd26003aebc5565ed5: Status 404 returned error can't find the container with id caabbbe25d6163c66a82e5346fa4461a7f4b2448e3fbbfcd26003aebc5565ed5
Jul 15 04:06:12.676235 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.676212    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ntm4b"
Jul 15 04:06:12.685169 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:12.685149    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b773b0_2e42_47c2_ab57_92a0162cf8db.slice/crio-400ed80debe701be46186a8012511ec184e8621dd1e36a5b5b23507078cef337 WatchSource:0}: Error finding container 400ed80debe701be46186a8012511ec184e8621dd1e36a5b5b23507078cef337: Status 404 returned error can't find the container with id 400ed80debe701be46186a8012511ec184e8621dd1e36a5b5b23507078cef337
Jul 15 04:06:12.699587 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.699565    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8hzpl"
Jul 15 04:06:12.704771 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.704753    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9f68m"
Jul 15 04:06:12.707665 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:12.707630    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01875d97_1469_4a47_a2df_2e43893ec711.slice/crio-eb8d5ab41acfb56854a8c9ed9c2c46aafb3e0eb492f5dfe9a82f0f0433ddb69c WatchSource:0}: Error finding container eb8d5ab41acfb56854a8c9ed9c2c46aafb3e0eb492f5dfe9a82f0f0433ddb69c: Status 404 returned error can't find the container with id eb8d5ab41acfb56854a8c9ed9c2c46aafb3e0eb492f5dfe9a82f0f0433ddb69c
Jul 15 04:06:12.713039 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:12.713012    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb4f7f2_d8c9_4451_ae39_97507126f420.slice/crio-de67b244437e424c401ba894f487f08c8b1d372a4bff2ca5149dc522211c4cbf WatchSource:0}: Error finding container de67b244437e424c401ba894f487f08c8b1d372a4bff2ca5149dc522211c4cbf: Status 404 returned error can't find the container with id de67b244437e424c401ba894f487f08c8b1d372a4bff2ca5149dc522211c4cbf
Jul 15 04:06:12.871763 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.871664    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twpb4\" (UniqueName: \"kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4\") pod \"network-check-target-txk5x\" (UID: \"5dcea033-1394-4680-b043-b24b294ccd7a\") " pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:12.871912 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.871830    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:06:12.871912 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.871852    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:06:12.871912 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.871862    2524 projected.go:196] Error preparing data for projected volume kube-api-access-twpb4 for pod openshift-network-diagnostics/network-check-target-txk5x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:12.872040 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.871916    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4 podName:5dcea033-1394-4680-b043-b24b294ccd7a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:13.871899218 +0000 UTC m=+3.183280445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-twpb4" (UniqueName: "kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4") pod "network-check-target-txk5x" (UID: "5dcea033-1394-4680-b043-b24b294ccd7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:12.972695 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.972661    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:12.972889 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:12.972722    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:12.972889 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.972806    2524 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:12.972889 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.972803    2524 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:12.972889 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.972866    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs podName:d62c63a3-e8fd-424a-811b-4e65e545f33a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:13.972852169 +0000 UTC m=+3.284233395 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs") pod "network-metrics-daemon-86pgb" (UID: "d62c63a3-e8fd-424a-811b-4e65e545f33a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:12.972889 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:12.972880    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret podName:bb2bdcea-aedf-4f70-8aae-b2901b9c4a83 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:13.972873473 +0000 UTC m=+3.284254699 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret") pod "global-pull-secret-syncer-swf7w" (UID: "bb2bdcea-aedf-4f70-8aae-b2901b9c4a83") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:13.051321 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:13.051272    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e94d729_dba7_454d_863f_2b9c091e65af.slice/crio-674c54ee8d9316c254b6e0a7c5545e7455543f49714960318fdcab9912bad947 WatchSource:0}: Error finding container 674c54ee8d9316c254b6e0a7c5545e7455543f49714960318fdcab9912bad947: Status 404 returned error can't find the container with id 674c54ee8d9316c254b6e0a7c5545e7455543f49714960318fdcab9912bad947
Jul 15 04:06:13.314416 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.314277    2524 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 04:01:12 +0000 UTC" deadline="2028-04-13 17:47:12.004870376 +0000 UTC"
Jul 15 04:06:13.314416 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.314317    2524 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15325h40m58.690557879s"
Jul 15 04:06:13.426374 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.426329    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:13.426529 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:13.426438    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:13.459860 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.459793    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrr2l" event={"ID":"04752185-efbf-4166-8b68-7834e27cb871","Type":"ContainerStarted","Data":"8cd02eaf1acf2ed981dc67b70232bd1e94990f42875845b3d631f6f70bc723a5"}
Jul 15 04:06:13.472553 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.472288    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d9v7v" event={"ID":"58f571a6-f29b-4885-bac5-8e637b7a2027","Type":"ContainerStarted","Data":"ad8d1ec3bc3c0dbe522c544d6e78ce5902040400844c695f4ea1423cc506cba1"}
Jul 15 04:06:13.483691 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.483637    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-76.ec2.internal" event={"ID":"f15257a2672d183728dc10e769d94d89","Type":"ContainerStarted","Data":"0a501846a4893731151db05f8059a0b9407d55713285ca89a8944df5c6011753"}
Jul 15 04:06:13.498623 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.498555    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg" event={"ID":"1e94d729-dba7-454d-863f-2b9c091e65af","Type":"ContainerStarted","Data":"674c54ee8d9316c254b6e0a7c5545e7455543f49714960318fdcab9912bad947"}
Jul 15 04:06:13.501899 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.501865    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9f68m" event={"ID":"afb4f7f2-d8c9-4451-ae39-97507126f420","Type":"ContainerStarted","Data":"de67b244437e424c401ba894f487f08c8b1d372a4bff2ca5149dc522211c4cbf"}
Jul 15 04:06:13.505599 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.505572    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kjdkn" event={"ID":"3d7355a2-bc8b-435e-8c31-60297ecc4e35","Type":"ContainerStarted","Data":"caabbbe25d6163c66a82e5346fa4461a7f4b2448e3fbbfcd26003aebc5565ed5"}
Jul 15 04:06:13.515276 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.515247    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc" event={"ID":"f853b65c-ad5d-4187-b60b-bdb5715a1873","Type":"ContainerStarted","Data":"12b696b4cff5c015f3c685ff3511f32ed7f7d7394c273f2c4666ca9dabc64e40"}
Jul 15 04:06:13.526977 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.526947    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zcjw5" event={"ID":"abfb7bf3-6f37-4fcd-9909-2483c2f5315f","Type":"ContainerStarted","Data":"2fa251dbef65f652429d22cbd479a0dd8b599d939f1be1848494c8b7d932644e"}
Jul 15 04:06:13.531011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.530982    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal" event={"ID":"6faa769590e33e316d4b91d47d08166c","Type":"ContainerStarted","Data":"c8b9a5354b936937c3f4ba6fa3a63a15b789bd95be2cdf66bbc3a1d71eec82ae"}
Jul 15 04:06:13.558764 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.558729    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8hzpl" event={"ID":"01875d97-1469-4a47-a2df-2e43893ec711","Type":"ContainerStarted","Data":"eb8d5ab41acfb56854a8c9ed9c2c46aafb3e0eb492f5dfe9a82f0f0433ddb69c"}
Jul 15 04:06:13.574557 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.574435    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ntm4b" event={"ID":"77b773b0-2e42-47c2-ab57-92a0162cf8db","Type":"ContainerStarted","Data":"400ed80debe701be46186a8012511ec184e8621dd1e36a5b5b23507078cef337"}
Jul 15 04:06:13.586331 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.586302    2524 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:06:13.750559 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.750379    2524 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:06:13.880496 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.880401    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twpb4\" (UniqueName: \"kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4\") pod \"network-check-target-txk5x\" (UID: \"5dcea033-1394-4680-b043-b24b294ccd7a\") " pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:13.880705 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:13.880581    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:06:13.880705 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:13.880605    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:06:13.880705 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:13.880619    2524 projected.go:196] Error preparing data for projected volume kube-api-access-twpb4 for pod openshift-network-diagnostics/network-check-target-txk5x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:13.880705 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:13.880700    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4 podName:5dcea033-1394-4680-b043-b24b294ccd7a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:15.880678814 +0000 UTC m=+5.192060064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-twpb4" (UniqueName: "kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4") pod "network-check-target-txk5x" (UID: "5dcea033-1394-4680-b043-b24b294ccd7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:13.991364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.981694    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:13.991364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:13.981766    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:13.991364 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:13.981855    2524 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:13.991364 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:13.981855    2524 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:13.991364 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:13.981914    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs podName:d62c63a3-e8fd-424a-811b-4e65e545f33a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:15.981894756 +0000 UTC m=+5.293275982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs") pod "network-metrics-daemon-86pgb" (UID: "d62c63a3-e8fd-424a-811b-4e65e545f33a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:13.991364 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:13.981932    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret podName:bb2bdcea-aedf-4f70-8aae-b2901b9c4a83 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:15.981922517 +0000 UTC m=+5.293303743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret") pod "global-pull-secret-syncer-swf7w" (UID: "bb2bdcea-aedf-4f70-8aae-b2901b9c4a83") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:14.314540 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:14.314415    2524 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 04:01:12 +0000 UTC" deadline="2028-03-05 01:51:01.020089897 +0000 UTC"
Jul 15 04:06:14.314540 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:14.314483    2524 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14373h44m46.705610952s"
Jul 15 04:06:14.426202 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:14.426166    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:14.426396 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:14.426308    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:14.426396 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:14.426391    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:14.426523 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:14.426467    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:15.428167 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:15.427594    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:15.428167 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:15.427801    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:15.903781 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:15.903685    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twpb4\" (UniqueName: \"kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4\") pod \"network-check-target-txk5x\" (UID: \"5dcea033-1394-4680-b043-b24b294ccd7a\") " pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:15.903976 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:15.903911    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:06:15.903976 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:15.903934    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:06:15.903976 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:15.903947    2524 projected.go:196] Error preparing data for projected volume kube-api-access-twpb4 for pod openshift-network-diagnostics/network-check-target-txk5x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:15.904133 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:15.904006    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4 podName:5dcea033-1394-4680-b043-b24b294ccd7a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:19.903988125 +0000 UTC m=+9.215369352 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-twpb4" (UniqueName: "kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4") pod "network-check-target-txk5x" (UID: "5dcea033-1394-4680-b043-b24b294ccd7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:16.005036 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:16.004291    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:16.005036 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:16.004370    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:16.005036 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:16.004534    2524 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:16.005036 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:16.004603    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs podName:d62c63a3-e8fd-424a-811b-4e65e545f33a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:20.004582623 +0000 UTC m=+9.315963852 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs") pod "network-metrics-daemon-86pgb" (UID: "d62c63a3-e8fd-424a-811b-4e65e545f33a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:16.005036 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:16.004632    2524 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:16.005036 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:16.004705    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret podName:bb2bdcea-aedf-4f70-8aae-b2901b9c4a83 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:20.004684339 +0000 UTC m=+9.316065580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret") pod "global-pull-secret-syncer-swf7w" (UID: "bb2bdcea-aedf-4f70-8aae-b2901b9c4a83") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:16.427274 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:16.427055    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:16.427458 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:16.427385    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:16.427798 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:16.427777    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:16.427938 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:16.427882    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:17.429245 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:17.426701    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:17.429756 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:17.429378    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:18.426810 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:18.426772    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:18.426985 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:18.426894    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:18.426985 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:18.426956    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:18.427091 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:18.427024    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:19.426735 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:19.426701    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:19.427186 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:19.426853    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:19.939490 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:19.939456    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twpb4\" (UniqueName: \"kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4\") pod \"network-check-target-txk5x\" (UID: \"5dcea033-1394-4680-b043-b24b294ccd7a\") " pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:19.939728 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:19.939702    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:06:19.939728 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:19.939724    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:06:19.939847 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:19.939737    2524 projected.go:196] Error preparing data for projected volume kube-api-access-twpb4 for pod openshift-network-diagnostics/network-check-target-txk5x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:19.939847 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:19.939791    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4 podName:5dcea033-1394-4680-b043-b24b294ccd7a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:27.939774088 +0000 UTC m=+17.251155316 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-twpb4" (UniqueName: "kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4") pod "network-check-target-txk5x" (UID: "5dcea033-1394-4680-b043-b24b294ccd7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:20.039924 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:20.039886    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:20.040109 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:20.039953    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:20.040109 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:20.040068    2524 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:20.040221 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:20.040128    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs podName:d62c63a3-e8fd-424a-811b-4e65e545f33a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:28.040109305 +0000 UTC m=+17.351490536 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs") pod "network-metrics-daemon-86pgb" (UID: "d62c63a3-e8fd-424a-811b-4e65e545f33a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:20.040513 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:20.040492    2524 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:20.040572 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:20.040552    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret podName:bb2bdcea-aedf-4f70-8aae-b2901b9c4a83 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:28.040536063 +0000 UTC m=+17.351917294 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret") pod "global-pull-secret-syncer-swf7w" (UID: "bb2bdcea-aedf-4f70-8aae-b2901b9c4a83") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:20.426633 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:20.426091    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:20.426633 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:20.426137    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:20.426633 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:20.426218    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:20.426633 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:20.426352    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:21.427405 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:21.427368    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:21.427898 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:21.427512    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:22.426352 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:22.426315    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:22.426509 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:22.426449    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:22.426890 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:22.426868    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:22.426991 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:22.426972    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:23.427771 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:23.427733    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:23.428182 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:23.427884    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:24.427113 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:24.427070    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:24.427113 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:24.427115    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:24.427356 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:24.427217    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:24.427356 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:24.427326    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:25.426215 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:25.426189    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:25.426541 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:25.426291    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:26.426352 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:26.426313    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:26.426836 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:26.426359    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:26.426836 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:26.426456    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:26.426836 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:26.426590    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:27.427092 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:27.427056    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:27.427546 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:27.427192    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:28.006011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:28.005957    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twpb4\" (UniqueName: \"kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4\") pod \"network-check-target-txk5x\" (UID: \"5dcea033-1394-4680-b043-b24b294ccd7a\") " pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:28.006210 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:28.006142    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:06:28.006210 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:28.006166    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:06:28.006210 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:28.006178    2524 projected.go:196] Error preparing data for projected volume kube-api-access-twpb4 for pod openshift-network-diagnostics/network-check-target-txk5x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:28.006362 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:28.006242    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4 podName:5dcea033-1394-4680-b043-b24b294ccd7a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:44.006224853 +0000 UTC m=+33.317606091 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-twpb4" (UniqueName: "kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4") pod "network-check-target-txk5x" (UID: "5dcea033-1394-4680-b043-b24b294ccd7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:28.107347 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:28.107311    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:28.107520 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:28.107357    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:28.107520 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:28.107469    2524 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:28.107520 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:28.107469    2524 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:28.107691 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:28.107536    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs podName:d62c63a3-e8fd-424a-811b-4e65e545f33a nodeName:}" failed. No retries permitted until 2026-07-15 04:06:44.107518111 +0000 UTC m=+33.418899337 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs") pod "network-metrics-daemon-86pgb" (UID: "d62c63a3-e8fd-424a-811b-4e65e545f33a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:28.107691 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:28.107553    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret podName:bb2bdcea-aedf-4f70-8aae-b2901b9c4a83 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:44.107544533 +0000 UTC m=+33.418925762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret") pod "global-pull-secret-syncer-swf7w" (UID: "bb2bdcea-aedf-4f70-8aae-b2901b9c4a83") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:28.426434 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:28.426354    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:28.426638 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:28.426458    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:28.426638 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:28.426368    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:28.426638 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:28.426524    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:29.427176 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:29.427138    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:29.427578 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:29.427256    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:30.426723 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:30.426691    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:30.426896 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:30.426694    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:30.426896 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:30.426806    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:30.426896 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:30.426865    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:31.427523 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:31.427493    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:31.427994 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:31.427643    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:32.426108 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:32.426074    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:32.426285 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:32.426074    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:32.426285 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:32.426206    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:32.426406 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:32.426289    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:33.427808 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:33.427767    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:33.428206 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:33.427917    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:34.426932 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.426707    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:34.427140 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:34.427088    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:34.427140 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.426766    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:34.427269 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:34.427216    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:34.648855 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.648827    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:06:34.649541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.649212    2524 generic.go:356] "Generic (PLEG): container finished" podID="f853b65c-ad5d-4187-b60b-bdb5715a1873" containerID="ea44276cdcefad47ce852c2942ac055aa87cfc7b7e075862833da84ca5cf3a64" exitCode=1
Jul 15 04:06:34.649541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.649290    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc" event={"ID":"f853b65c-ad5d-4187-b60b-bdb5715a1873","Type":"ContainerStarted","Data":"14c62d63dd4bc291f239992e08ebde2d06d74178dcb87d8a96e5c8cd4ba79199"}
Jul 15 04:06:34.649541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.649351    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc" event={"ID":"f853b65c-ad5d-4187-b60b-bdb5715a1873","Type":"ContainerStarted","Data":"d1017c84270241bee6ef3462bb6df3dd43127791d01a0a390d5433c5613363f3"}
Jul 15 04:06:34.649541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.649380    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc" event={"ID":"f853b65c-ad5d-4187-b60b-bdb5715a1873","Type":"ContainerStarted","Data":"b95f0f8d9530a1d675261709c4eb74765ce5e888d676ea15051b31feee86a110"}
Jul 15 04:06:34.649541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.649407    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc" event={"ID":"f853b65c-ad5d-4187-b60b-bdb5715a1873","Type":"ContainerStarted","Data":"f541fe853a49361ed4366a06649c22fb5134047a9dae4d75fb9495e42aa8913b"}
Jul 15 04:06:34.649541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.649432    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc" event={"ID":"f853b65c-ad5d-4187-b60b-bdb5715a1873","Type":"ContainerDied","Data":"ea44276cdcefad47ce852c2942ac055aa87cfc7b7e075862833da84ca5cf3a64"}
Jul 15 04:06:34.649541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.649459    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc" event={"ID":"f853b65c-ad5d-4187-b60b-bdb5715a1873","Type":"ContainerStarted","Data":"adad0a580fa0dad51984c6354d583d6b8d6f78a81efde197e595c037c66156f9"}
Jul 15 04:06:34.651252 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.651198    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zcjw5" event={"ID":"abfb7bf3-6f37-4fcd-9909-2483c2f5315f","Type":"ContainerStarted","Data":"d91758db91cbbeca9a6c0e3e69965d38f3a503c1a5f1ff4f0cf5b7e398ace170"}
Jul 15 04:06:34.652732 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.652697    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ntm4b" event={"ID":"77b773b0-2e42-47c2-ab57-92a0162cf8db","Type":"ContainerStarted","Data":"85cbb22cb147d99b4ea1b509d253478a0c128dd18aa6159befef3197185e5163"}
Jul 15 04:06:34.654528 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.654505    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-76.ec2.internal" event={"ID":"f15257a2672d183728dc10e769d94d89","Type":"ContainerStarted","Data":"5ce94a1be13783e7015511eaf511882a704f0f8e306c41776a03ecac0d12cb2f"}
Jul 15 04:06:34.669667 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.669576    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zcjw5" podStartSLOduration=2.412093915 podStartE2EDuration="23.669559909s" totalImagesPullingTime="21.257465994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:34.668884913 +0000 UTC m=+23.980266159" watchObservedRunningTime="2026-07-15 04:06:34.669559909 +0000 UTC m=+23.980941154"
Jul 15 04:06:34.684846 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.684797    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-76.ec2.internal" podStartSLOduration=22.68478079 podStartE2EDuration="22.68478079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:34.684442655 +0000 UTC m=+23.995823901" watchObservedRunningTime="2026-07-15 04:06:34.68478079 +0000 UTC m=+23.996162036"
Jul 15 04:06:34.699714 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:34.699661    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ntm4b" podStartSLOduration=2.512020995 podStartE2EDuration="23.699643604s" totalImagesPullingTime="21.187622609s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:34.69950721 +0000 UTC m=+24.010888456" watchObservedRunningTime="2026-07-15 04:06:34.699643604 +0000 UTC m=+24.011024850"
Jul 15 04:06:35.427094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.426821    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:35.427250 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:35.427223    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:35.658427 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.658332    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8hzpl" event={"ID":"01875d97-1469-4a47-a2df-2e43893ec711","Type":"ContainerStarted","Data":"32bf66165bb14dfc10510d5b6397de9a03e800878a7e67839daf05055df9229a"}
Jul 15 04:06:35.660102 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.660072    2524 generic.go:356] "Generic (PLEG): container finished" podID="04752185-efbf-4166-8b68-7834e27cb871" containerID="eeab091d9fe316275dc9fd998053b77a0c1c8ed9261cfc0b10dbf3451b5c26af" exitCode=0
Jul 15 04:06:35.660232 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.660162    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrr2l" event={"ID":"04752185-efbf-4166-8b68-7834e27cb871","Type":"ContainerDied","Data":"eeab091d9fe316275dc9fd998053b77a0c1c8ed9261cfc0b10dbf3451b5c26af"}
Jul 15 04:06:35.661815 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.661681    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d9v7v" event={"ID":"58f571a6-f29b-4885-bac5-8e637b7a2027","Type":"ContainerStarted","Data":"ba5b246d6acc1da951c4badc30e8333653929bb1bc898983bf8507e8d36aa270"}
Jul 15 04:06:35.663406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.663380    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg" event={"ID":"1e94d729-dba7-454d-863f-2b9c091e65af","Type":"ContainerStarted","Data":"9e4ae28e7df6bf238bb951285c1aeb5c0d8cc1447a2e9c5d9e2e88e4e257cd22"}
Jul 15 04:06:35.664900 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.664873    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9f68m" event={"ID":"afb4f7f2-d8c9-4451-ae39-97507126f420","Type":"ContainerStarted","Data":"c895938e9bb75b13b633b8edab654faf97cce7d609d0cbfbf0251dee32ddc5a5"}
Jul 15 04:06:35.666850 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.666806    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kjdkn" event={"ID":"3d7355a2-bc8b-435e-8c31-60297ecc4e35","Type":"ContainerStarted","Data":"39afe66f38fff891829de0191a061f8a150cd3dc5efd6933eaa035e0d2dba508"}
Jul 15 04:06:35.668401 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.668378    2524 generic.go:356] "Generic (PLEG): container finished" podID="6faa769590e33e316d4b91d47d08166c" containerID="35700c6855bf25c658896831cd5cb6e6d833840192f8bd1a76f50946757297fe" exitCode=0
Jul 15 04:06:35.668580 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.668554    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal" event={"ID":"6faa769590e33e316d4b91d47d08166c","Type":"ContainerDied","Data":"35700c6855bf25c658896831cd5cb6e6d833840192f8bd1a76f50946757297fe"}
Jul 15 04:06:35.688830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.688789    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8hzpl" podStartSLOduration=2.539080775 podStartE2EDuration="23.688775239s" totalImagesPullingTime="21.149694464s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:35.674808121 +0000 UTC m=+24.986189366" watchObservedRunningTime="2026-07-15 04:06:35.688775239 +0000 UTC m=+25.000156487"
Jul 15 04:06:35.742758 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.742715    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-d9v7v" podStartSLOduration=3.519200625 podStartE2EDuration="24.742700774s" totalImagesPullingTime="21.223500149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:35.742472587 +0000 UTC m=+25.053853832" watchObservedRunningTime="2026-07-15 04:06:35.742700774 +0000 UTC m=+25.054082020"
Jul 15 04:06:35.742897 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.742795    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9f68m" podStartSLOduration=2.612295785 podStartE2EDuration="23.742792018s" totalImagesPullingTime="21.130496233s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:35.728310223 +0000 UTC m=+25.039691480" watchObservedRunningTime="2026-07-15 04:06:35.742792018 +0000 UTC m=+25.054173282"
Jul 15 04:06:35.755859 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:35.755816    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kjdkn" podStartSLOduration=3.552295239 podStartE2EDuration="24.755804224s" totalImagesPullingTime="21.203508985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:35.755536269 +0000 UTC m=+25.066917515" watchObservedRunningTime="2026-07-15 04:06:35.755804224 +0000 UTC m=+25.067185468"
Jul 15 04:06:36.153042 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:36.153019    2524 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 04:06:36.323961 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:36.323809    2524 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T04:06:36.15303662Z","UUID":"fb7c23c4-bac4-4446-a2cd-a733c4ba9bc5","Handler":null,"Name":"","Endpoint":""}
Jul 15 04:06:36.332468 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:36.332440    2524 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 04:06:36.332640 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:36.332508    2524 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 04:06:36.426690 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:36.426660    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:36.426860 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:36.426781    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:36.426860 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:36.426841    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:36.426995 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:36.426958    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:36.673547 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:36.673441    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg" event={"ID":"1e94d729-dba7-454d-863f-2b9c091e65af","Type":"ContainerStarted","Data":"2f99c9569c4a104cd3d571d7fca328a54fbc83fd742b45200e92045e69184896"}
Jul 15 04:06:36.675728 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:36.675696    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal" event={"ID":"6faa769590e33e316d4b91d47d08166c","Type":"ContainerStarted","Data":"6b927b4303d6c583b792195083755dde03fa0e18ac92b331a9fbfbf0b6cc0820"}
Jul 15 04:06:36.691879 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:36.691838    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-76.ec2.internal" podStartSLOduration=24.691823532 podStartE2EDuration="24.691823532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:36.691646589 +0000 UTC m=+26.003027836" watchObservedRunningTime="2026-07-15 04:06:36.691823532 +0000 UTC m=+26.003204782"
Jul 15 04:06:37.426185 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:37.426156    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:37.426333 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:37.426296    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:37.680532 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:37.680494    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg" event={"ID":"1e94d729-dba7-454d-863f-2b9c091e65af","Type":"ContainerStarted","Data":"6ea51887234f4f6ba78f355fd7ee8045bf79b5fb199b0ac75e46c146639fbb7a"}
Jul 15 04:06:37.684529 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:37.684509    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:06:37.685009 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:37.684972    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc" event={"ID":"f853b65c-ad5d-4187-b60b-bdb5715a1873","Type":"ContainerStarted","Data":"3a72dc6ca34ec7b37d78029aad773736b6f95cb020cd5ebcc4e0723001b7fb80"}
Jul 15 04:06:37.699027 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:37.698974    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkszg" podStartSLOduration=3.4576507960000002 podStartE2EDuration="26.698954502s" totalImagesPullingTime="23.241303706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:37.698817525 +0000 UTC m=+27.010198773" watchObservedRunningTime="2026-07-15 04:06:37.698954502 +0000 UTC m=+27.010335749"
Jul 15 04:06:37.983048 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:37.982790    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:37.983356 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:37.983334    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:38.426539 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:38.426501    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:38.426539 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:38.426532    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:38.426805 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:38.426650    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:38.426859 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:38.426803    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:39.426995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:39.426962    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:39.427542 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:39.427089    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:40.426929 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.426744    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:40.427068 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.426794    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:40.427068 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:40.427012    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:40.427556 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:40.427086    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:40.590468 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.590436    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:40.590632 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.590553    2524 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 15 04:06:40.591011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.590995    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d9v7v"
Jul 15 04:06:40.695413 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.695339    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:06:40.695764 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.695735    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc" event={"ID":"f853b65c-ad5d-4187-b60b-bdb5715a1873","Type":"ContainerStarted","Data":"697911e6d1b3ecad049594054b3b9f970379b9447c293f200bdb52391836a2d6"}
Jul 15 04:06:40.695999 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.695973    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:40.696211 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.696194    2524 scope.go:117] "RemoveContainer" containerID="ea44276cdcefad47ce852c2942ac055aa87cfc7b7e075862833da84ca5cf3a64"
Jul 15 04:06:40.697562 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.697538    2524 generic.go:356] "Generic (PLEG): container finished" podID="04752185-efbf-4166-8b68-7834e27cb871" containerID="67ffe889a96ad797099a186f05c2782cdd6044eca3fc80de5c3b7cddd26164d7" exitCode=0
Jul 15 04:06:40.697694 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.697640    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrr2l" event={"ID":"04752185-efbf-4166-8b68-7834e27cb871","Type":"ContainerDied","Data":"67ffe889a96ad797099a186f05c2782cdd6044eca3fc80de5c3b7cddd26164d7"}
Jul 15 04:06:40.713149 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:40.713129    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:41.427565 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.427528    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:41.428065 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:41.427670    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:41.704312 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.704287    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:06:41.704787 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.704749    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc" event={"ID":"f853b65c-ad5d-4187-b60b-bdb5715a1873","Type":"ContainerStarted","Data":"593dd23e2b14a1d365ff9add19d5b273637d24b0b75be49e17ee632b63d6bf9d"}
Jul 15 04:06:41.705044 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.705026    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:41.705239 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.705199    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:41.720367 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.720342    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:06:41.733984 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.733934    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc" podStartSLOduration=9.455865159 podStartE2EDuration="30.733920776s" totalImagesPullingTime="21.278055617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:41.733729681 +0000 UTC m=+31.045110925" watchObservedRunningTime="2026-07-15 04:06:41.733920776 +0000 UTC m=+31.045302021"
Jul 15 04:06:41.875254 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.875217    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-txk5x"]
Jul 15 04:06:41.875385 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.875347    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:41.875478 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:41.875453    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:41.878265 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.878239    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-swf7w"]
Jul 15 04:06:41.878393 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.878380    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:41.878511 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:41.878491    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:41.886829 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.886807    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-86pgb"]
Jul 15 04:06:41.886977 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:41.886894    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:41.886977 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:41.886964    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:42.709315 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:42.709275    2524 generic.go:356] "Generic (PLEG): container finished" podID="04752185-efbf-4166-8b68-7834e27cb871" containerID="52a8fbe6b183de6f6705527790f052e56299eeeac2c78f97a03de1b06e3450c0" exitCode=0
Jul 15 04:06:42.709764 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:42.709325    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrr2l" event={"ID":"04752185-efbf-4166-8b68-7834e27cb871","Type":"ContainerDied","Data":"52a8fbe6b183de6f6705527790f052e56299eeeac2c78f97a03de1b06e3450c0"}
Jul 15 04:06:43.426902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:43.426863    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:43.427116 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:43.426918    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:43.427116 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:43.426873    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:43.427116 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:43.427005    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:43.427116 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:43.427049    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:43.427283 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:43.427151    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:43.714032 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:43.713955    2524 generic.go:356] "Generic (PLEG): container finished" podID="04752185-efbf-4166-8b68-7834e27cb871" containerID="5911ea9fc7f4ef8f9db0ec131b76b16f82ed487b0e43054a9b4c012dc0914bd8" exitCode=0
Jul 15 04:06:43.714569 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:43.714038    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrr2l" event={"ID":"04752185-efbf-4166-8b68-7834e27cb871","Type":"ContainerDied","Data":"5911ea9fc7f4ef8f9db0ec131b76b16f82ed487b0e43054a9b4c012dc0914bd8"}
Jul 15 04:06:44.033168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:44.033085    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twpb4\" (UniqueName: \"kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4\") pod \"network-check-target-txk5x\" (UID: \"5dcea033-1394-4680-b043-b24b294ccd7a\") " pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:44.033306 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:44.033204    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:06:44.033306 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:44.033219    2524 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:06:44.033306 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:44.033228    2524 projected.go:196] Error preparing data for projected volume kube-api-access-twpb4 for pod openshift-network-diagnostics/network-check-target-txk5x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:44.033306 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:44.033278    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4 podName:5dcea033-1394-4680-b043-b24b294ccd7a nodeName:}" failed. No retries permitted until 2026-07-15 04:07:16.033265902 +0000 UTC m=+65.344647127 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-twpb4" (UniqueName: "kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4") pod "network-check-target-txk5x" (UID: "5dcea033-1394-4680-b043-b24b294ccd7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:06:44.134131 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:44.134095    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:44.134313 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:44.134143    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:44.134313 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:44.134261    2524 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:44.134313 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:44.134272    2524 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:44.134439 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:44.134335    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret podName:bb2bdcea-aedf-4f70-8aae-b2901b9c4a83 nodeName:}" failed. No retries permitted until 2026-07-15 04:07:16.134316013 +0000 UTC m=+65.445697238 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret") pod "global-pull-secret-syncer-swf7w" (UID: "bb2bdcea-aedf-4f70-8aae-b2901b9c4a83") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:06:44.134439 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:44.134356    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs podName:d62c63a3-e8fd-424a-811b-4e65e545f33a nodeName:}" failed. No retries permitted until 2026-07-15 04:07:16.134345117 +0000 UTC m=+65.445726346 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs") pod "network-metrics-daemon-86pgb" (UID: "d62c63a3-e8fd-424a-811b-4e65e545f33a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:06:45.426310 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:45.426278    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:45.426887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:45.426280    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:45.426887 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:45.426401    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-txk5x" podUID="5dcea033-1394-4680-b043-b24b294ccd7a"
Jul 15 04:06:45.426887 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:45.426491    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-86pgb" podUID="d62c63a3-e8fd-424a-811b-4e65e545f33a"
Jul 15 04:06:45.426887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:45.426291    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:45.426887 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:45.426572    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swf7w" podUID="bb2bdcea-aedf-4f70-8aae-b2901b9c4a83"
Jul 15 04:06:46.961903 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:46.961874    2524 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-76.ec2.internal" event="NodeReady"
Jul 15 04:06:46.962415 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:46.961999    2524 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 04:06:46.999691 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:46.999653    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5458f57594-5xkzm"]
Jul 15 04:06:47.028031 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.027144    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h7bmp"]
Jul 15 04:06:47.028031 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.027294    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.030237 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.030213    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 04:06:47.030376 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.030268    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 04:06:47.030376 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.030319    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 04:06:47.030376 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.030325    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r7tdt\""
Jul 15 04:06:47.037578 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.037552    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 04:06:47.043147 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.043128    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-44srg"]
Jul 15 04:06:47.043260 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.043231    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.046633 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.046613    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 04:06:47.046745 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.046650    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2wv7r\""
Jul 15 04:06:47.046861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.046845    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 04:06:47.065233 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.065213    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5458f57594-5xkzm"]
Jul 15 04:06:47.065360 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.065248    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-44srg"]
Jul 15 04:06:47.065360 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.065267    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h7bmp"]
Jul 15 04:06:47.065360 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.065360    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-44srg"
Jul 15 04:06:47.067963 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.067936    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-j29sx\""
Jul 15 04:06:47.068091 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.067947    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 04:06:47.068323 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.068296    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 04:06:47.069322 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.069307    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 04:06:47.155664 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.155633    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-image-registry-private-configuration\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.155664 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.155666    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-ca-trust-extracted\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.155888 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.155694    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-installation-pull-secrets\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.155888 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.155775    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-config-volume\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.155888 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.155810    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.155888 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.155829    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcqs4\" (UniqueName: \"kubernetes.io/projected/52b6b8d4-5eee-4b5d-96cb-befdb1d9b901-kube-api-access-jcqs4\") pod \"ingress-canary-44srg\" (UID: \"52b6b8d4-5eee-4b5d-96cb-befdb1d9b901\") " pod="openshift-ingress-canary/ingress-canary-44srg"
Jul 15 04:06:47.155888 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.155851    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjzjt\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-kube-api-access-sjzjt\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.155888 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.155888    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-tmp-dir\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.156168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.155923    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52b6b8d4-5eee-4b5d-96cb-befdb1d9b901-cert\") pod \"ingress-canary-44srg\" (UID: \"52b6b8d4-5eee-4b5d-96cb-befdb1d9b901\") " pod="openshift-ingress-canary/ingress-canary-44srg"
Jul 15 04:06:47.156168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.155952    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-certificates\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.156168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.155995    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-trusted-ca\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.156168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.156045    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-bound-sa-token\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.156168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.156075    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.156168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.156099    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsft9\" (UniqueName: \"kubernetes.io/projected/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-kube-api-access-lsft9\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.257301 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257222    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-config-volume\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.257301 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257260    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.257301 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257283    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcqs4\" (UniqueName: \"kubernetes.io/projected/52b6b8d4-5eee-4b5d-96cb-befdb1d9b901-kube-api-access-jcqs4\") pod \"ingress-canary-44srg\" (UID: \"52b6b8d4-5eee-4b5d-96cb-befdb1d9b901\") " pod="openshift-ingress-canary/ingress-canary-44srg"
Jul 15 04:06:47.257568 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257312    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjzjt\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-kube-api-access-sjzjt\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.257568 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257344    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-tmp-dir\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.257568 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257380    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52b6b8d4-5eee-4b5d-96cb-befdb1d9b901-cert\") pod \"ingress-canary-44srg\" (UID: \"52b6b8d4-5eee-4b5d-96cb-befdb1d9b901\") " pod="openshift-ingress-canary/ingress-canary-44srg"
Jul 15 04:06:47.257568 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:47.257409    2524 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 04:06:47.257568 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:47.257433    2524 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5458f57594-5xkzm: secret "image-registry-tls" not found
Jul 15 04:06:47.257568 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:47.257500    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls podName:bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:47.757481613 +0000 UTC m=+37.068862842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls") pod "image-registry-5458f57594-5xkzm" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3") : secret "image-registry-tls" not found
Jul 15 04:06:47.257902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257417    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-certificates\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.257902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257702    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-trusted-ca\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.257902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257727    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-tmp-dir\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.257902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257740    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-bound-sa-token\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.257902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257775    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.257902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257795    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsft9\" (UniqueName: \"kubernetes.io/projected/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-kube-api-access-lsft9\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.257902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257822    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-image-registry-private-configuration\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.257902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257840    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-ca-trust-extracted\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.257902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257860    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-installation-pull-secrets\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.258300 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.257907    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-config-volume\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.258300 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.258058    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-certificates\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.258300 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:47.258198    2524 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:06:47.258300 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:47.258253    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls podName:a6b85d0d-2ca6-408b-b00d-60edb7fb41c7 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:47.758238154 +0000 UTC m=+37.069619381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls") pod "dns-default-h7bmp" (UID: "a6b85d0d-2ca6-408b-b00d-60edb7fb41c7") : secret "dns-default-metrics-tls" not found
Jul 15 04:06:47.258426 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.258315    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-ca-trust-extracted\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.258567 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.258501    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-trusted-ca\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.261906 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.261885    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52b6b8d4-5eee-4b5d-96cb-befdb1d9b901-cert\") pod \"ingress-canary-44srg\" (UID: \"52b6b8d4-5eee-4b5d-96cb-befdb1d9b901\") " pod="openshift-ingress-canary/ingress-canary-44srg"
Jul 15 04:06:47.262023 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.261914    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-image-registry-private-configuration\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.262262 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.262241    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-installation-pull-secrets\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.266294 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.266249    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-bound-sa-token\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.266520 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.266496    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcqs4\" (UniqueName: \"kubernetes.io/projected/52b6b8d4-5eee-4b5d-96cb-befdb1d9b901-kube-api-access-jcqs4\") pod \"ingress-canary-44srg\" (UID: \"52b6b8d4-5eee-4b5d-96cb-befdb1d9b901\") " pod="openshift-ingress-canary/ingress-canary-44srg"
Jul 15 04:06:47.266991 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.266970    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjzjt\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-kube-api-access-sjzjt\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.267154 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.267136    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsft9\" (UniqueName: \"kubernetes.io/projected/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-kube-api-access-lsft9\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.374796 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.374759    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-44srg"
Jul 15 04:06:47.426227 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.426193    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:06:47.426393 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.426201    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:06:47.426456 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.426204    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:06:47.429935 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.429898    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 04:06:47.430063 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.429941    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 04:06:47.430063 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.429906    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8rjwx\""
Jul 15 04:06:47.430063 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.429998    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 04:06:47.430063 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.429938    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 04:06:47.430063 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.429944    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nc2gd\""
Jul 15 04:06:47.762478 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.762271    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:47.762649 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.762517    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:47.762649 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:47.762415    2524 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:06:47.762761 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:47.762654    2524 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 04:06:47.762761 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:47.762667    2524 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5458f57594-5xkzm: secret "image-registry-tls" not found
Jul 15 04:06:47.762761 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:47.762672    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls podName:a6b85d0d-2ca6-408b-b00d-60edb7fb41c7 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:48.762652544 +0000 UTC m=+38.074033773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls") pod "dns-default-h7bmp" (UID: "a6b85d0d-2ca6-408b-b00d-60edb7fb41c7") : secret "dns-default-metrics-tls" not found
Jul 15 04:06:47.762761 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:47.762700    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls podName:bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:48.762689213 +0000 UTC m=+38.074070437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls") pod "image-registry-5458f57594-5xkzm" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3") : secret "image-registry-tls" not found
Jul 15 04:06:47.782354 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:47.782328    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-44srg"]
Jul 15 04:06:47.838641 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:47.838578    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b6b8d4_5eee_4b5d_96cb_befdb1d9b901.slice/crio-7d52f53ff1c541ac3e0d34868c84666299832fdf72fff3191894f99cb4d5263e WatchSource:0}: Error finding container 7d52f53ff1c541ac3e0d34868c84666299832fdf72fff3191894f99cb4d5263e: Status 404 returned error can't find the container with id 7d52f53ff1c541ac3e0d34868c84666299832fdf72fff3191894f99cb4d5263e
Jul 15 04:06:48.728314 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:48.728277    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-44srg" event={"ID":"52b6b8d4-5eee-4b5d-96cb-befdb1d9b901","Type":"ContainerStarted","Data":"7d52f53ff1c541ac3e0d34868c84666299832fdf72fff3191894f99cb4d5263e"}
Jul 15 04:06:48.731344 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:48.731313    2524 generic.go:356] "Generic (PLEG): container finished" podID="04752185-efbf-4166-8b68-7834e27cb871" containerID="e853689ca669bcd5b590e4a3baf2104dee13efb90a26aab0cd88dc82ef504d92" exitCode=0
Jul 15 04:06:48.731490 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:48.731358    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrr2l" event={"ID":"04752185-efbf-4166-8b68-7834e27cb871","Type":"ContainerDied","Data":"e853689ca669bcd5b590e4a3baf2104dee13efb90a26aab0cd88dc82ef504d92"}
Jul 15 04:06:48.769568 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:48.769526    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:48.769775 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:48.769656    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:48.769832 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:48.769816    2524 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:06:48.769895 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:48.769874    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls podName:a6b85d0d-2ca6-408b-b00d-60edb7fb41c7 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:50.769856595 +0000 UTC m=+40.081237820 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls") pod "dns-default-h7bmp" (UID: "a6b85d0d-2ca6-408b-b00d-60edb7fb41c7") : secret "dns-default-metrics-tls" not found
Jul 15 04:06:48.770283 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:48.770261    2524 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 04:06:48.770283 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:48.770284    2524 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5458f57594-5xkzm: secret "image-registry-tls" not found
Jul 15 04:06:48.770463 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:48.770328    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls podName:bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:50.770314103 +0000 UTC m=+40.081695330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls") pod "image-registry-5458f57594-5xkzm" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3") : secret "image-registry-tls" not found
Jul 15 04:06:49.737366 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:49.737332    2524 generic.go:356] "Generic (PLEG): container finished" podID="04752185-efbf-4166-8b68-7834e27cb871" containerID="4557f35415d637cb3fd6314d030800e1587667f262064f396bed03d46c9ec069" exitCode=0
Jul 15 04:06:49.737824 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:49.737402    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrr2l" event={"ID":"04752185-efbf-4166-8b68-7834e27cb871","Type":"ContainerDied","Data":"4557f35415d637cb3fd6314d030800e1587667f262064f396bed03d46c9ec069"}
Jul 15 04:06:50.741757 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:50.741487    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-44srg" event={"ID":"52b6b8d4-5eee-4b5d-96cb-befdb1d9b901","Type":"ContainerStarted","Data":"3d42c585b7df9d3f627ec83c71750e686f4a82feedaab5cff55c407d4636d2f6"}
Jul 15 04:06:50.744811 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:50.744788    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrr2l" event={"ID":"04752185-efbf-4166-8b68-7834e27cb871","Type":"ContainerStarted","Data":"2911a4ae6901b7eb857e809a879f690883fd76b43f439bf2c4cda838bb2753f0"}
Jul 15 04:06:50.756570 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:50.756525    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-44srg" podStartSLOduration=2.447756684 podStartE2EDuration="4.756512253s" totalImagesPullingTime="2.308755569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:50.755888052 +0000 UTC m=+40.067269311" watchObservedRunningTime="2026-07-15 04:06:50.756512253 +0000 UTC m=+40.067893498"
Jul 15 04:06:50.776741 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:50.776692    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vrr2l" podStartSLOduration=8.086826232 podStartE2EDuration="39.776676462s" totalImagesPullingTime="31.68985023s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:06:50.775197597 +0000 UTC m=+40.086578842" watchObservedRunningTime="2026-07-15 04:06:50.776676462 +0000 UTC m=+40.088057760"
Jul 15 04:06:50.788311 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:50.788291    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:50.788417 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:50.788336    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:50.788496 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:50.788431    2524 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 04:06:50.788496 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:50.788437    2524 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:06:50.788496 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:50.788443    2524 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5458f57594-5xkzm: secret "image-registry-tls" not found
Jul 15 04:06:50.788578 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:50.788498    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls podName:a6b85d0d-2ca6-408b-b00d-60edb7fb41c7 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:54.788481984 +0000 UTC m=+44.099863210 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls") pod "dns-default-h7bmp" (UID: "a6b85d0d-2ca6-408b-b00d-60edb7fb41c7") : secret "dns-default-metrics-tls" not found
Jul 15 04:06:50.788578 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:50.788512    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls podName:bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3 nodeName:}" failed. No retries permitted until 2026-07-15 04:06:54.78850662 +0000 UTC m=+44.099887844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls") pod "image-registry-5458f57594-5xkzm" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3") : secret "image-registry-tls" not found
Jul 15 04:06:54.818762 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:54.818721    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:06:54.819290 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:54.818782    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:06:54.819290 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:54.818872    2524 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:06:54.819290 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:54.818939    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls podName:a6b85d0d-2ca6-408b-b00d-60edb7fb41c7 nodeName:}" failed. No retries permitted until 2026-07-15 04:07:02.818923998 +0000 UTC m=+52.130305222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls") pod "dns-default-h7bmp" (UID: "a6b85d0d-2ca6-408b-b00d-60edb7fb41c7") : secret "dns-default-metrics-tls" not found
Jul 15 04:06:54.819290 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:54.818874    2524 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 04:06:54.819290 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:54.818976    2524 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5458f57594-5xkzm: secret "image-registry-tls" not found
Jul 15 04:06:54.819290 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:54.819022    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls podName:bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3 nodeName:}" failed. No retries permitted until 2026-07-15 04:07:02.819010506 +0000 UTC m=+52.130391730 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls") pod "image-registry-5458f57594-5xkzm" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3") : secret "image-registry-tls" not found
Jul 15 04:06:54.950667 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:54.950637    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-slcc9"]
Jul 15 04:06:54.972075 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:54.972049    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:54.976202 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:54.976177    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 04:06:54.976308 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:54.976291    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 04:06:54.976757 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:54.976743    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 04:06:54.977295 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:54.977275    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 04:06:54.977392 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:54.977339    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tzsqp\""
Jul 15 04:06:54.981905 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:54.981882    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-slcc9"]
Jul 15 04:06:55.020414 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.020382    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbsfz\" (UniqueName: \"kubernetes.io/projected/c9bef378-e519-4b5a-b871-e37044c90eae-kube-api-access-wbsfz\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.020564 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.020447    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9bef378-e519-4b5a-b871-e37044c90eae-data-volume\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.020564 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.020472    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.020564 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.020507    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.020564 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.020554    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c9bef378-e519-4b5a-b871-e37044c90eae-crio-socket\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.121567 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.121535    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.121760 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.121620    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c9bef378-e519-4b5a-b871-e37044c90eae-crio-socket\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.121760 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.121669    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbsfz\" (UniqueName: \"kubernetes.io/projected/c9bef378-e519-4b5a-b871-e37044c90eae-kube-api-access-wbsfz\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.121760 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:55.121716    2524 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 04:06:55.121760 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.121753    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c9bef378-e519-4b5a-b871-e37044c90eae-crio-socket\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.121920 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.121724    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9bef378-e519-4b5a-b871-e37044c90eae-data-volume\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.121920 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:55.121783    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm podName:c9bef378-e519-4b5a-b871-e37044c90eae nodeName:}" failed. No retries permitted until 2026-07-15 04:06:55.621766236 +0000 UTC m=+44.933147460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-slcc9" (UID: "c9bef378-e519-4b5a-b871-e37044c90eae") : configmap "kube-rbac-proxy" not found
Jul 15 04:06:55.121920 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.121850    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.122011 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:55.121951    2524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 04:06:55.122011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.121980    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9bef378-e519-4b5a-b871-e37044c90eae-data-volume\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.122068 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:55.122016    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls podName:c9bef378-e519-4b5a-b871-e37044c90eae nodeName:}" failed. No retries permitted until 2026-07-15 04:06:55.622000815 +0000 UTC m=+44.933382058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-slcc9" (UID: "c9bef378-e519-4b5a-b871-e37044c90eae") : secret "insights-runtime-extractor-tls" not found
Jul 15 04:06:55.133536 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.133509    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbsfz\" (UniqueName: \"kubernetes.io/projected/c9bef378-e519-4b5a-b871-e37044c90eae-kube-api-access-wbsfz\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.625904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.625875    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.626039 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:55.625916    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:55.626039 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:55.626008    2524 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 04:06:55.626118 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:55.626037    2524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 04:06:55.626118 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:55.626061    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm podName:c9bef378-e519-4b5a-b871-e37044c90eae nodeName:}" failed. No retries permitted until 2026-07-15 04:06:56.626048773 +0000 UTC m=+45.937429997 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-slcc9" (UID: "c9bef378-e519-4b5a-b871-e37044c90eae") : configmap "kube-rbac-proxy" not found
Jul 15 04:06:55.626118 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:55.626086    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls podName:c9bef378-e519-4b5a-b871-e37044c90eae nodeName:}" failed. No retries permitted until 2026-07-15 04:06:56.626074397 +0000 UTC m=+45.937455621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-slcc9" (UID: "c9bef378-e519-4b5a-b871-e37044c90eae") : secret "insights-runtime-extractor-tls" not found
Jul 15 04:06:56.634528 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:56.634492    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:56.634528 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:56.634536    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:56.635016 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:56.634643    2524 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 04:06:56.635016 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:56.634695    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm podName:c9bef378-e519-4b5a-b871-e37044c90eae nodeName:}" failed. No retries permitted until 2026-07-15 04:06:58.634682289 +0000 UTC m=+47.946063513 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-slcc9" (UID: "c9bef378-e519-4b5a-b871-e37044c90eae") : configmap "kube-rbac-proxy" not found
Jul 15 04:06:56.635016 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:56.634693    2524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 04:06:56.635016 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:56.634762    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls podName:c9bef378-e519-4b5a-b871-e37044c90eae nodeName:}" failed. No retries permitted until 2026-07-15 04:06:58.634744452 +0000 UTC m=+47.946125677 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-slcc9" (UID: "c9bef378-e519-4b5a-b871-e37044c90eae") : secret "insights-runtime-extractor-tls" not found
Jul 15 04:06:56.838855 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:56.838830    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kjdkn_3d7355a2-bc8b-435e-8c31-60297ecc4e35/dns-node-resolver/0.log"
Jul 15 04:06:57.837875 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:57.837847    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9f68m_afb4f7f2-d8c9-4451-ae39-97507126f420/node-ca/0.log"
Jul 15 04:06:57.970111 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:57.970079    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-45gql"]
Jul 15 04:06:57.989360 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:57.989337    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-45gql"]
Jul 15 04:06:57.989488 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:57.989420    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-45gql"
Jul 15 04:06:57.992046 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:57.992023    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 15 04:06:57.992046 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:57.992040    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 15 04:06:57.992881 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:57.992865    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 15 04:06:57.992955 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:57.992893    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rf2fh\""
Jul 15 04:06:57.993014 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:57.992960    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 15 04:06:58.044081 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.044042    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/01965c6b-0f39-42c7-b33b-4d492d86b1e4-signing-key\") pod \"service-ca-79f68b9f4f-45gql\" (UID: \"01965c6b-0f39-42c7-b33b-4d492d86b1e4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45gql"
Jul 15 04:06:58.044262 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.044167    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/01965c6b-0f39-42c7-b33b-4d492d86b1e4-signing-cabundle\") pod \"service-ca-79f68b9f4f-45gql\" (UID: \"01965c6b-0f39-42c7-b33b-4d492d86b1e4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45gql"
Jul 15 04:06:58.044262 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.044206    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpjbv\" (UniqueName: \"kubernetes.io/projected/01965c6b-0f39-42c7-b33b-4d492d86b1e4-kube-api-access-hpjbv\") pod \"service-ca-79f68b9f4f-45gql\" (UID: \"01965c6b-0f39-42c7-b33b-4d492d86b1e4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45gql"
Jul 15 04:06:58.145426 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.145385    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/01965c6b-0f39-42c7-b33b-4d492d86b1e4-signing-key\") pod \"service-ca-79f68b9f4f-45gql\" (UID: \"01965c6b-0f39-42c7-b33b-4d492d86b1e4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45gql"
Jul 15 04:06:58.145565 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.145496    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/01965c6b-0f39-42c7-b33b-4d492d86b1e4-signing-cabundle\") pod \"service-ca-79f68b9f4f-45gql\" (UID: \"01965c6b-0f39-42c7-b33b-4d492d86b1e4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45gql"
Jul 15 04:06:58.145565 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.145529    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpjbv\" (UniqueName: \"kubernetes.io/projected/01965c6b-0f39-42c7-b33b-4d492d86b1e4-kube-api-access-hpjbv\") pod \"service-ca-79f68b9f4f-45gql\" (UID: \"01965c6b-0f39-42c7-b33b-4d492d86b1e4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45gql"
Jul 15 04:06:58.146248 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.146228    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/01965c6b-0f39-42c7-b33b-4d492d86b1e4-signing-cabundle\") pod \"service-ca-79f68b9f4f-45gql\" (UID: \"01965c6b-0f39-42c7-b33b-4d492d86b1e4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45gql"
Jul 15 04:06:58.147946 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.147927    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/01965c6b-0f39-42c7-b33b-4d492d86b1e4-signing-key\") pod \"service-ca-79f68b9f4f-45gql\" (UID: \"01965c6b-0f39-42c7-b33b-4d492d86b1e4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45gql"
Jul 15 04:06:58.153575 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.153545    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpjbv\" (UniqueName: \"kubernetes.io/projected/01965c6b-0f39-42c7-b33b-4d492d86b1e4-kube-api-access-hpjbv\") pod \"service-ca-79f68b9f4f-45gql\" (UID: \"01965c6b-0f39-42c7-b33b-4d492d86b1e4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-45gql"
Jul 15 04:06:58.298413 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.298371    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-45gql"
Jul 15 04:06:58.424338 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.424275    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-45gql"]
Jul 15 04:06:58.427066 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:06:58.427035    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01965c6b_0f39_42c7_b33b_4d492d86b1e4.slice/crio-98a3d3f55fa502c78227e19402350a2ad22b964f76d08c88745a3c7f3783b1e7 WatchSource:0}: Error finding container 98a3d3f55fa502c78227e19402350a2ad22b964f76d08c88745a3c7f3783b1e7: Status 404 returned error can't find the container with id 98a3d3f55fa502c78227e19402350a2ad22b964f76d08c88745a3c7f3783b1e7
Jul 15 04:06:58.642508 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.642476    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-44srg_52b6b8d4-5eee-4b5d-96cb-befdb1d9b901/serve-healthcheck-canary/0.log"
Jul 15 04:06:58.649777 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.649749    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:58.649889 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.649793    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:06:58.649926 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:58.649909    2524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 04:06:58.649955 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:58.649925    2524 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 04:06:58.649986 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:58.649975    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls podName:c9bef378-e519-4b5a-b871-e37044c90eae nodeName:}" failed. No retries permitted until 2026-07-15 04:07:02.64995773 +0000 UTC m=+51.961338956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-slcc9" (UID: "c9bef378-e519-4b5a-b871-e37044c90eae") : secret "insights-runtime-extractor-tls" not found
Jul 15 04:06:58.650028 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:06:58.649989    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm podName:c9bef378-e519-4b5a-b871-e37044c90eae nodeName:}" failed. No retries permitted until 2026-07-15 04:07:02.649982546 +0000 UTC m=+51.961363771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-slcc9" (UID: "c9bef378-e519-4b5a-b871-e37044c90eae") : configmap "kube-rbac-proxy" not found
Jul 15 04:06:58.764525 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:06:58.764435    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-45gql" event={"ID":"01965c6b-0f39-42c7-b33b-4d492d86b1e4","Type":"ContainerStarted","Data":"98a3d3f55fa502c78227e19402350a2ad22b964f76d08c88745a3c7f3783b1e7"}
Jul 15 04:07:01.773047 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:01.773011    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-45gql" event={"ID":"01965c6b-0f39-42c7-b33b-4d492d86b1e4","Type":"ContainerStarted","Data":"a6bdfb65b815c5f4b8ca4f51799d529bbbe644ea44d998cc3753daac2d1771a6"}
Jul 15 04:07:01.787190 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:01.787135    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-45gql" podStartSLOduration=2.326656238 podStartE2EDuration="4.787119476s" totalImagesPullingTime="2.460463238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:06:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:01.786353675 +0000 UTC m=+51.097734920" watchObservedRunningTime="2026-07-15 04:07:01.787119476 +0000 UTC m=+51.098500722"
Jul 15 04:07:02.683050 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:02.683005    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:07:02.683254 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:02.683058    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:07:02.683254 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:02.683178    2524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 04:07:02.683254 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:02.683191    2524 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 04:07:02.683254 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:02.683241    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm podName:c9bef378-e519-4b5a-b871-e37044c90eae nodeName:}" failed. No retries permitted until 2026-07-15 04:07:10.683222496 +0000 UTC m=+59.994603722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-slcc9" (UID: "c9bef378-e519-4b5a-b871-e37044c90eae") : configmap "kube-rbac-proxy" not found
Jul 15 04:07:02.683459 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:02.683259    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls podName:c9bef378-e519-4b5a-b871-e37044c90eae nodeName:}" failed. No retries permitted until 2026-07-15 04:07:10.683248869 +0000 UTC m=+59.994630093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-slcc9" (UID: "c9bef378-e519-4b5a-b871-e37044c90eae") : secret "insights-runtime-extractor-tls" not found
Jul 15 04:07:02.884703 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:02.884671    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:07:02.885242 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:02.884826    2524 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:07:02.885365 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:02.885212    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:07:02.885365 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:02.885284    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls podName:a6b85d0d-2ca6-408b-b00d-60edb7fb41c7 nodeName:}" failed. No retries permitted until 2026-07-15 04:07:18.885264999 +0000 UTC m=+68.196646230 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls") pod "dns-default-h7bmp" (UID: "a6b85d0d-2ca6-408b-b00d-60edb7fb41c7") : secret "dns-default-metrics-tls" not found
Jul 15 04:07:02.885365 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:02.885289    2524 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 04:07:02.885365 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:02.885304    2524 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5458f57594-5xkzm: secret "image-registry-tls" not found
Jul 15 04:07:02.885365 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:02.885346    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls podName:bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3 nodeName:}" failed. No retries permitted until 2026-07-15 04:07:18.885332033 +0000 UTC m=+68.196713264 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls") pod "image-registry-5458f57594-5xkzm" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3") : secret "image-registry-tls" not found
Jul 15 04:07:10.752167 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:10.752133    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:07:10.752167 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:10.752175    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:07:10.752769 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:10.752290    2524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 04:07:10.752769 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:10.752354    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls podName:c9bef378-e519-4b5a-b871-e37044c90eae nodeName:}" failed. No retries permitted until 2026-07-15 04:07:26.752337815 +0000 UTC m=+76.063719039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-slcc9" (UID: "c9bef378-e519-4b5a-b871-e37044c90eae") : secret "insights-runtime-extractor-tls" not found
Jul 15 04:07:10.752769 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:10.752621    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9bef378-e519-4b5a-b871-e37044c90eae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:07:13.724974 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:13.724946    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kmwrc"
Jul 15 04:07:16.092296 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.092240    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twpb4\" (UniqueName: \"kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4\") pod \"network-check-target-txk5x\" (UID: \"5dcea033-1394-4680-b043-b24b294ccd7a\") " pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:07:16.094991 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.094970    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 04:07:16.105061 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.105029    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 04:07:16.116305 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.116282    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twpb4\" (UniqueName: \"kubernetes.io/projected/5dcea033-1394-4680-b043-b24b294ccd7a-kube-api-access-twpb4\") pod \"network-check-target-txk5x\" (UID: \"5dcea033-1394-4680-b043-b24b294ccd7a\") " pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:07:16.192924 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.192880    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:07:16.193102 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.192960    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:07:16.195700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.195682    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 04:07:16.196216 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.196201    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 04:07:16.206233 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.206213    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bb2bdcea-aedf-4f70-8aae-b2901b9c4a83-original-pull-secret\") pod \"global-pull-secret-syncer-swf7w\" (UID: \"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83\") " pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:07:16.206327 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.206244    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62c63a3-e8fd-424a-811b-4e65e545f33a-metrics-certs\") pod \"network-metrics-daemon-86pgb\" (UID: \"d62c63a3-e8fd-424a-811b-4e65e545f33a\") " pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:07:16.241047 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.241024    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nc2gd\""
Jul 15 04:07:16.244142 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.244129    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swf7w"
Jul 15 04:07:16.248855 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.248835    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:07:16.252281 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.252258    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8rjwx\""
Jul 15 04:07:16.260168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.260149    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-86pgb"
Jul 15 04:07:16.388425 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.388393    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-txk5x"]
Jul 15 04:07:16.391362 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:16.391335    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcea033_1394_4680_b043_b24b294ccd7a.slice/crio-3f67e912e08e50286ad3604cbca833397a804098adff56562c0017734129fd33 WatchSource:0}: Error finding container 3f67e912e08e50286ad3604cbca833397a804098adff56562c0017734129fd33: Status 404 returned error can't find the container with id 3f67e912e08e50286ad3604cbca833397a804098adff56562c0017734129fd33
Jul 15 04:07:16.608882 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.608808    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-swf7w"]
Jul 15 04:07:16.611825 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:16.611793    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2bdcea_aedf_4f70_8aae_b2901b9c4a83.slice/crio-7bfe8649f1c53beea7b930270473c7c0bbece1b76be286de0d662c4e1b236875 WatchSource:0}: Error finding container 7bfe8649f1c53beea7b930270473c7c0bbece1b76be286de0d662c4e1b236875: Status 404 returned error can't find the container with id 7bfe8649f1c53beea7b930270473c7c0bbece1b76be286de0d662c4e1b236875
Jul 15 04:07:16.612018 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.612000    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-86pgb"]
Jul 15 04:07:16.617722 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:16.617686    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd62c63a3_e8fd_424a_811b_4e65e545f33a.slice/crio-78cfb797c4c4d65bfb9e89def5918dcac88debe72523c4c15959b0dae86cd5fe WatchSource:0}: Error finding container 78cfb797c4c4d65bfb9e89def5918dcac88debe72523c4c15959b0dae86cd5fe: Status 404 returned error can't find the container with id 78cfb797c4c4d65bfb9e89def5918dcac88debe72523c4c15959b0dae86cd5fe
Jul 15 04:07:16.809534 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.809497    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-86pgb" event={"ID":"d62c63a3-e8fd-424a-811b-4e65e545f33a","Type":"ContainerStarted","Data":"78cfb797c4c4d65bfb9e89def5918dcac88debe72523c4c15959b0dae86cd5fe"}
Jul 15 04:07:16.810565 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.810542    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-txk5x" event={"ID":"5dcea033-1394-4680-b043-b24b294ccd7a","Type":"ContainerStarted","Data":"3f67e912e08e50286ad3604cbca833397a804098adff56562c0017734129fd33"}
Jul 15 04:07:16.811517 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:16.811498    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-swf7w" event={"ID":"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83","Type":"ContainerStarted","Data":"7bfe8649f1c53beea7b930270473c7c0bbece1b76be286de0d662c4e1b236875"}
Jul 15 04:07:17.818474 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:17.818436    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-86pgb" event={"ID":"d62c63a3-e8fd-424a-811b-4e65e545f33a","Type":"ContainerStarted","Data":"e398931d347a91f181fdfdaa678e6fd8cf92a8ad4c3069f81f8c682ffd6e882c"}
Jul 15 04:07:17.818937 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:17.818504    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-86pgb" event={"ID":"d62c63a3-e8fd-424a-811b-4e65e545f33a","Type":"ContainerStarted","Data":"7014b2285eaa8deda32e9a99ede8119bdba5d34097fc903309014cbadefdebcc"}
Jul 15 04:07:17.836784 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:17.836640    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-86pgb" podStartSLOduration=64.881961306 podStartE2EDuration="1m5.836624605s" totalImagesPullingTime="954.663299ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:17.835873293 +0000 UTC m=+67.147254541" watchObservedRunningTime="2026-07-15 04:07:17.836624605 +0000 UTC m=+67.148005866"
Jul 15 04:07:18.854896 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.854863    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28"]
Jul 15 04:07:18.857799 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.857778    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28"
Jul 15 04:07:18.861645 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.861619    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 15 04:07:18.861862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.861845    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-g84zh\""
Jul 15 04:07:18.861953 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.861935    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 04:07:18.862176 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.862161    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 04:07:18.862341 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.862328    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 04:07:18.869924 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.869902    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28"]
Jul 15 04:07:18.911133 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.911093    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5458f57594-5xkzm"]
Jul 15 04:07:18.911340 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:18.911319    2524 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5458f57594-5xkzm" podUID="bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3"
Jul 15 04:07:18.914156 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.914126    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:07:18.914285 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.914171    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/95aa6757-1bc8-4031-8251-d43133799f44-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-588ffcd5f4-8kd28\" (UID: \"95aa6757-1bc8-4031-8251-d43133799f44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28"
Jul 15 04:07:18.914381 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.914360    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:07:18.914440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.914419    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7kqb\" (UniqueName: \"kubernetes.io/projected/95aa6757-1bc8-4031-8251-d43133799f44-kube-api-access-h7kqb\") pod \"managed-serviceaccount-addon-agent-588ffcd5f4-8kd28\" (UID: \"95aa6757-1bc8-4031-8251-d43133799f44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28"
Jul 15 04:07:18.917132 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.917113    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6b85d0d-2ca6-408b-b00d-60edb7fb41c7-metrics-tls\") pod \"dns-default-h7bmp\" (UID: \"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7\") " pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:07:18.917238 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.917167    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls\") pod \"image-registry-5458f57594-5xkzm\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") " pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:07:18.956639 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.956518    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-ht4kn"]
Jul 15 04:07:18.958520 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.958493    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-ht4kn"
Jul 15 04:07:18.962326 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.962295    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 04:07:18.962852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.962825    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 04:07:18.963130 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.963105    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-87wlt\""
Jul 15 04:07:18.965632 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.965587    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-696df47dcf-x5cn2"]
Jul 15 04:07:18.968240 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.968221    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:18.981721 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.981699    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-ht4kn"]
Jul 15 04:07:18.995680 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:18.995657    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-696df47dcf-x5cn2"]
Jul 15 04:07:19.015781 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.015746    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/95aa6757-1bc8-4031-8251-d43133799f44-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-588ffcd5f4-8kd28\" (UID: \"95aa6757-1bc8-4031-8251-d43133799f44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28"
Jul 15 04:07:19.015940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.015809    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b925e347-51a5-425f-a836-6333d707ae6d-trusted-ca\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.015940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.015838    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b925e347-51a5-425f-a836-6333d707ae6d-bound-sa-token\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.015940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.015867    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7kqb\" (UniqueName: \"kubernetes.io/projected/95aa6757-1bc8-4031-8251-d43133799f44-kube-api-access-h7kqb\") pod \"managed-serviceaccount-addon-agent-588ffcd5f4-8kd28\" (UID: \"95aa6757-1bc8-4031-8251-d43133799f44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28"
Jul 15 04:07:19.015940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.015894    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b925e347-51a5-425f-a836-6333d707ae6d-installation-pull-secrets\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.015940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.015918    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlbcc\" (UniqueName: \"kubernetes.io/projected/93c2aa95-043a-4825-b306-f64202da3c9c-kube-api-access-hlbcc\") pod \"downloads-68c9cbb497-ht4kn\" (UID: \"93c2aa95-043a-4825-b306-f64202da3c9c\") " pod="openshift-console/downloads-68c9cbb497-ht4kn"
Jul 15 04:07:19.016168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.015950    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b925e347-51a5-425f-a836-6333d707ae6d-registry-certificates\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.016168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.015977    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggbz8\" (UniqueName: \"kubernetes.io/projected/b925e347-51a5-425f-a836-6333d707ae6d-kube-api-access-ggbz8\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.016168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.016033    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b925e347-51a5-425f-a836-6333d707ae6d-registry-tls\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.016168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.016115    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b925e347-51a5-425f-a836-6333d707ae6d-image-registry-private-configuration\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.016168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.016146    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b925e347-51a5-425f-a836-6333d707ae6d-ca-trust-extracted\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.018959 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.018938    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/95aa6757-1bc8-4031-8251-d43133799f44-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-588ffcd5f4-8kd28\" (UID: \"95aa6757-1bc8-4031-8251-d43133799f44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28"
Jul 15 04:07:19.035314 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.035286    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7kqb\" (UniqueName: \"kubernetes.io/projected/95aa6757-1bc8-4031-8251-d43133799f44-kube-api-access-h7kqb\") pod \"managed-serviceaccount-addon-agent-588ffcd5f4-8kd28\" (UID: \"95aa6757-1bc8-4031-8251-d43133799f44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28"
Jul 15 04:07:19.117258 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.117170    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b925e347-51a5-425f-a836-6333d707ae6d-ca-trust-extracted\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.117427 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.117282    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b925e347-51a5-425f-a836-6333d707ae6d-trusted-ca\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.117427 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.117326    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b925e347-51a5-425f-a836-6333d707ae6d-bound-sa-token\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.117427 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.117349    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b925e347-51a5-425f-a836-6333d707ae6d-installation-pull-secrets\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.117427 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.117375    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlbcc\" (UniqueName: \"kubernetes.io/projected/93c2aa95-043a-4825-b306-f64202da3c9c-kube-api-access-hlbcc\") pod \"downloads-68c9cbb497-ht4kn\" (UID: \"93c2aa95-043a-4825-b306-f64202da3c9c\") " pod="openshift-console/downloads-68c9cbb497-ht4kn"
Jul 15 04:07:19.117427 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.117414    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b925e347-51a5-425f-a836-6333d707ae6d-registry-certificates\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.117800 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.117455    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggbz8\" (UniqueName: \"kubernetes.io/projected/b925e347-51a5-425f-a836-6333d707ae6d-kube-api-access-ggbz8\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.117800 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.117499    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b925e347-51a5-425f-a836-6333d707ae6d-registry-tls\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.117800 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.117545    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b925e347-51a5-425f-a836-6333d707ae6d-image-registry-private-configuration\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.117800 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.117666    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b925e347-51a5-425f-a836-6333d707ae6d-ca-trust-extracted\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.118334 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.118299    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b925e347-51a5-425f-a836-6333d707ae6d-registry-certificates\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.118566 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.118536    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b925e347-51a5-425f-a836-6333d707ae6d-trusted-ca\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.120239 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.120218    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b925e347-51a5-425f-a836-6333d707ae6d-installation-pull-secrets\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.120343 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.120323    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b925e347-51a5-425f-a836-6333d707ae6d-registry-tls\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.120522 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.120503    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b925e347-51a5-425f-a836-6333d707ae6d-image-registry-private-configuration\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.126319 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.126294    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggbz8\" (UniqueName: \"kubernetes.io/projected/b925e347-51a5-425f-a836-6333d707ae6d-kube-api-access-ggbz8\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.126873 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.126844    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlbcc\" (UniqueName: \"kubernetes.io/projected/93c2aa95-043a-4825-b306-f64202da3c9c-kube-api-access-hlbcc\") pod \"downloads-68c9cbb497-ht4kn\" (UID: \"93c2aa95-043a-4825-b306-f64202da3c9c\") " pod="openshift-console/downloads-68c9cbb497-ht4kn"
Jul 15 04:07:19.130686 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.130665    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b925e347-51a5-425f-a836-6333d707ae6d-bound-sa-token\") pod \"image-registry-696df47dcf-x5cn2\" (UID: \"b925e347-51a5-425f-a836-6333d707ae6d\") " pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.154895 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.154869    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2wv7r\""
Jul 15 04:07:19.162922 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.162892    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:07:19.177794 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.177763    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28"
Jul 15 04:07:19.270436 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.270395    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-ht4kn"
Jul 15 04:07:19.282697 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.282673    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r7tdt\""
Jul 15 04:07:19.290789 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.290756    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:19.824876 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.824847    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:07:19.831061 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.831038    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:07:19.925146 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.925110    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-installation-pull-secrets\") pod \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") "
Jul 15 04:07:19.925553 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.925155    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjzjt\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-kube-api-access-sjzjt\") pod \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") "
Jul 15 04:07:19.925553 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.925181    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-trusted-ca\") pod \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") "
Jul 15 04:07:19.925553 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.925209    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls\") pod \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") "
Jul 15 04:07:19.925737 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.925681    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-bound-sa-token\") pod \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") "
Jul 15 04:07:19.925791 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.925768    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-image-registry-private-configuration\") pod \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") "
Jul 15 04:07:19.925843 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.925814    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-certificates\") pod \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") "
Jul 15 04:07:19.925898 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.925847    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-ca-trust-extracted\") pod \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\" (UID: \"bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3\") "
Jul 15 04:07:19.925947 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.925907    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:07:19.926416 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.926164    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:07:19.926416 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.926335    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:07:19.926416 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.926214    2524 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-trusted-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:07:19.928357 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.928314    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:07:19.928464 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.928353    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:07:19.928464 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.928380    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:07:19.929633 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.929573    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-kube-api-access-sjzjt" (OuterVolumeSpecName: "kube-api-access-sjzjt") pod "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3"). InnerVolumeSpecName "kube-api-access-sjzjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:07:19.929886 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:19.929850    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3" (UID: "bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:07:20.027359 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:20.027321    2524 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-certificates\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:07:20.027359 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:20.027360    2524 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-ca-trust-extracted\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:07:20.027576 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:20.027372    2524 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-installation-pull-secrets\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:07:20.027576 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:20.027380    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjzjt\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-kube-api-access-sjzjt\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:07:20.027576 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:20.027391    2524 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-registry-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:07:20.027576 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:20.027400    2524 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-bound-sa-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:07:20.027576 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:20.027408    2524 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3-image-registry-private-configuration\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:07:20.827444 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:20.827418    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5458f57594-5xkzm"
Jul 15 04:07:20.860703 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:20.860669    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5458f57594-5xkzm"]
Jul 15 04:07:20.864416 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:20.864393    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5458f57594-5xkzm"]
Jul 15 04:07:21.194318 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.194240    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-ht4kn"]
Jul 15 04:07:21.197091 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:21.197063    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c2aa95_043a_4825_b306_f64202da3c9c.slice/crio-3bfa253e7afef4ce95555c5fc43c3727bf88525777da1b4f3ceb8014ae7d6b28 WatchSource:0}: Error finding container 3bfa253e7afef4ce95555c5fc43c3727bf88525777da1b4f3ceb8014ae7d6b28: Status 404 returned error can't find the container with id 3bfa253e7afef4ce95555c5fc43c3727bf88525777da1b4f3ceb8014ae7d6b28
Jul 15 04:07:21.396319 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.396288    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28"]
Jul 15 04:07:21.399274 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:21.399237    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95aa6757_1bc8_4031_8251_d43133799f44.slice/crio-8667b12525cd9282623eb6bb67a9ecccf7f26b62ef4ae9884f786a3f1a9baf84 WatchSource:0}: Error finding container 8667b12525cd9282623eb6bb67a9ecccf7f26b62ef4ae9884f786a3f1a9baf84: Status 404 returned error can't find the container with id 8667b12525cd9282623eb6bb67a9ecccf7f26b62ef4ae9884f786a3f1a9baf84
Jul 15 04:07:21.401486 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.401465    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h7bmp"]
Jul 15 04:07:21.404431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.404410    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-696df47dcf-x5cn2"]
Jul 15 04:07:21.409451 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:21.409427    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb925e347_51a5_425f_a836_6333d707ae6d.slice/crio-c21e70cfdd0b77e8e38fa304ba3016d45de7d56009a30c21e060926275c7eaac WatchSource:0}: Error finding container c21e70cfdd0b77e8e38fa304ba3016d45de7d56009a30c21e060926275c7eaac: Status 404 returned error can't find the container with id c21e70cfdd0b77e8e38fa304ba3016d45de7d56009a30c21e060926275c7eaac
Jul 15 04:07:21.433646 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.433621    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3" path="/var/lib/kubelet/pods/bc1ed7ae-102e-46ef-8b9f-db5f5dc6f7e3/volumes"
Jul 15 04:07:21.833622 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.833515    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-txk5x" event={"ID":"5dcea033-1394-4680-b043-b24b294ccd7a","Type":"ContainerStarted","Data":"3c5caa6e793a8e89870c15ae172f7986ed555598c501f6fd09468cc858a1383e"}
Jul 15 04:07:21.834372 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.834319    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:07:21.836610 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.836563    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-swf7w" event={"ID":"bb2bdcea-aedf-4f70-8aae-b2901b9c4a83","Type":"ContainerStarted","Data":"fde0c8acdc5392ce354fd9d1c68486cd2b86fd66e23a451a3873ccd5a0eb8e7e"}
Jul 15 04:07:21.838480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.838424    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7bmp" event={"ID":"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7","Type":"ContainerStarted","Data":"825c077880e24e5fa0df692228af2b98229c72fa9bd42bed992183014e7788b4"}
Jul 15 04:07:21.840312 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.840282    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-696df47dcf-x5cn2" event={"ID":"b925e347-51a5-425f-a836-6333d707ae6d","Type":"ContainerStarted","Data":"29ced763e6dd29d3a8cd8dfe7d28af5239bfb2087ef01abbc4984b17b66569e4"}
Jul 15 04:07:21.840414 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.840330    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-696df47dcf-x5cn2" event={"ID":"b925e347-51a5-425f-a836-6333d707ae6d","Type":"ContainerStarted","Data":"c21e70cfdd0b77e8e38fa304ba3016d45de7d56009a30c21e060926275c7eaac"}
Jul 15 04:07:21.840479 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.840448    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:21.841734 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.841712    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28" event={"ID":"95aa6757-1bc8-4031-8251-d43133799f44","Type":"ContainerStarted","Data":"8667b12525cd9282623eb6bb67a9ecccf7f26b62ef4ae9884f786a3f1a9baf84"}
Jul 15 04:07:21.843584 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.843555    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-ht4kn" event={"ID":"93c2aa95-043a-4825-b306-f64202da3c9c","Type":"ContainerStarted","Data":"3bfa253e7afef4ce95555c5fc43c3727bf88525777da1b4f3ceb8014ae7d6b28"}
Jul 15 04:07:21.853780 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.853729    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-txk5x" podStartSLOduration=66.245865118 podStartE2EDuration="1m10.853714908s" totalImagesPullingTime="4.60784979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:21.853407842 +0000 UTC m=+71.164789090" watchObservedRunningTime="2026-07-15 04:07:21.853714908 +0000 UTC m=+71.165096154"
Jul 15 04:07:21.871882 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.871802    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-696df47dcf-x5cn2" podStartSLOduration=3.871782113 podStartE2EDuration="3.871782113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:21.871200046 +0000 UTC m=+71.182581292" watchObservedRunningTime="2026-07-15 04:07:21.871782113 +0000 UTC m=+71.183163360"
Jul 15 04:07:21.888429 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:21.887707    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-swf7w" podStartSLOduration=65.487559426 podStartE2EDuration="1m9.887686607s" totalImagesPullingTime="4.400127181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:21.887305019 +0000 UTC m=+71.198686265" watchObservedRunningTime="2026-07-15 04:07:21.887686607 +0000 UTC m=+71.199067855"
Jul 15 04:07:22.266606 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:22.266570    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk"]
Jul 15 04:07:22.306814 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:22.306783    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk"]
Jul 15 04:07:22.306965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:22.306886    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk"
Jul 15 04:07:22.309468 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:22.309444    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 15 04:07:22.309794 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:22.309769    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jt49r\""
Jul 15 04:07:22.446460 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:22.446423    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ptjnk\" (UID: \"a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk"
Jul 15 04:07:22.549670 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:22.547898    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ptjnk\" (UID: \"a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk"
Jul 15 04:07:22.549670 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:22.548054    2524 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 15 04:07:22.549670 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:22.548120    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0-tls-certificates podName:a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0 nodeName:}" failed. No retries permitted until 2026-07-15 04:07:23.048100279 +0000 UTC m=+72.359481509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-ptjnk" (UID: "a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 15 04:07:23.052276 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:23.052236    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ptjnk\" (UID: \"a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk"
Jul 15 04:07:23.055811 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:23.055782    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ptjnk\" (UID: \"a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk"
Jul 15 04:07:23.216373 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:23.216333    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk"
Jul 15 04:07:24.386172 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.386141    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58499467bd-km6kx"]
Jul 15 04:07:24.389177 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.388392    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.392033 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.392011    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gw7xd\""
Jul 15 04:07:24.392145 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.392039    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 04:07:24.392145 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.392138    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 04:07:24.392266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.392044    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 04:07:24.392316 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.392275    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 04:07:24.392372 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.392322    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 04:07:24.401507 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.401486    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58499467bd-km6kx"]
Jul 15 04:07:24.462041 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.461996    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5fng\" (UniqueName: \"kubernetes.io/projected/3579a35b-cc97-4c1e-986d-979b1220e357-kube-api-access-s5fng\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.462041 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.462043    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-service-ca\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.462302 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.462120    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-serving-cert\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.462302 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.462154    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-oauth-serving-cert\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.462302 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.462219    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-oauth-config\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.462302 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.462265    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-console-config\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.563237 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.563199    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5fng\" (UniqueName: \"kubernetes.io/projected/3579a35b-cc97-4c1e-986d-979b1220e357-kube-api-access-s5fng\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.563416 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.563248    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-service-ca\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.563416 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.563300    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-serving-cert\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.563416 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.563322    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-oauth-serving-cert\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.563416 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.563341    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-oauth-config\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.563416 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.563358    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-console-config\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.564155 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.564131    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-oauth-serving-cert\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.564205 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.564131    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-console-config\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.564268 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.564241    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-service-ca\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.566904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.566859    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-oauth-config\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.567022 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.566982    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-serving-cert\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.574835 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.574812    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5fng\" (UniqueName: \"kubernetes.io/projected/3579a35b-cc97-4c1e-986d-979b1220e357-kube-api-access-s5fng\") pod \"console-58499467bd-km6kx\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") " pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.704203 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.704170    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:24.798469 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.798378    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk"]
Jul 15 04:07:24.802662 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:24.802624    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8e6a017_a7a6_4bd3_bd99_0043a1c28cd0.slice/crio-bb82dde93a981fb5abb17126ff152611518dd5d113d2ef1171b27c65c96ed3fa WatchSource:0}: Error finding container bb82dde93a981fb5abb17126ff152611518dd5d113d2ef1171b27c65c96ed3fa: Status 404 returned error can't find the container with id bb82dde93a981fb5abb17126ff152611518dd5d113d2ef1171b27c65c96ed3fa
Jul 15 04:07:24.863191 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.862578    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7bmp" event={"ID":"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7","Type":"ContainerStarted","Data":"66e7e502a9673ad913e3273b36c063bfb04e38f4fd58ae7ec1cc66d5c76fde65"}
Jul 15 04:07:24.865692 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.864157    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28" event={"ID":"95aa6757-1bc8-4031-8251-d43133799f44","Type":"ContainerStarted","Data":"87c5f17823a1bb2547328403e273a7e1e2b0a96c896e939d6b89db0e51330bb0"}
Jul 15 04:07:24.867399 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.867337    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk" event={"ID":"a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0","Type":"ContainerStarted","Data":"bb82dde93a981fb5abb17126ff152611518dd5d113d2ef1171b27c65c96ed3fa"}
Jul 15 04:07:24.880970 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.880921    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588ffcd5f4-8kd28" podStartSLOduration=3.60284104 podStartE2EDuration="6.880906406s" totalImagesPullingTime="3.278065366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:24.879928735 +0000 UTC m=+74.191309981" watchObservedRunningTime="2026-07-15 04:07:24.880906406 +0000 UTC m=+74.192287630"
Jul 15 04:07:24.892668 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:24.892582    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58499467bd-km6kx"]
Jul 15 04:07:24.893433 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:24.893412    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3579a35b_cc97_4c1e_986d_979b1220e357.slice/crio-3fd417a89d98e524bd5b85221338133bbca0c2546d889dffbba1602f88b9e4e0 WatchSource:0}: Error finding container 3fd417a89d98e524bd5b85221338133bbca0c2546d889dffbba1602f88b9e4e0: Status 404 returned error can't find the container with id 3fd417a89d98e524bd5b85221338133bbca0c2546d889dffbba1602f88b9e4e0
Jul 15 04:07:25.873944 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:25.873899    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7bmp" event={"ID":"a6b85d0d-2ca6-408b-b00d-60edb7fb41c7","Type":"ContainerStarted","Data":"4a7a001da8f1b74ef00b879fb4644058ee4467f45403feb9ae16e5409785b616"}
Jul 15 04:07:25.874400 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:25.874117    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:07:25.875412 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:25.875366    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58499467bd-km6kx" event={"ID":"3579a35b-cc97-4c1e-986d-979b1220e357","Type":"ContainerStarted","Data":"3fd417a89d98e524bd5b85221338133bbca0c2546d889dffbba1602f88b9e4e0"}
Jul 15 04:07:26.373755 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.372486    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-h7bmp" podStartSLOduration=37.1081927 podStartE2EDuration="40.372464593s" totalImagesPullingTime="3.264271893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:25.890816788 +0000 UTC m=+75.202198034" watchObservedRunningTime="2026-07-15 04:07:26.372464593 +0000 UTC m=+75.683845841"
Jul 15 04:07:26.373755 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.372674    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-954b68c7f-njw74"]
Jul 15 04:07:26.375991 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.375970    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.385709 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.385460    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-954b68c7f-njw74"]
Jul 15 04:07:26.385709 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.385518    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 04:07:26.478477 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.478387    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-serving-cert\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.478477 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.478435    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-config\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.478477 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.478466    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k87hb\" (UniqueName: \"kubernetes.io/projected/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-kube-api-access-k87hb\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.478756 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.478491    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-trusted-ca-bundle\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.478756 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.478538    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-oauth-config\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.478756 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.478584    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-oauth-serving-cert\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.478756 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.478631    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-service-ca\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.579398 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.579357    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-oauth-serving-cert\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.579398 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.579405    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-service-ca\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.579688 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.579459    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-serving-cert\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.579688 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.579475    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-config\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.579688 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.579490    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k87hb\" (UniqueName: \"kubernetes.io/projected/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-kube-api-access-k87hb\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.579688 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.579507    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-trusted-ca-bundle\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.579688 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.579529    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-oauth-config\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.580178 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.580150    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-oauth-serving-cert\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.580343 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.580326    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-config\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.580819 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.580795    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-service-ca\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.581574 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.581554    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-trusted-ca-bundle\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.582630 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.582585    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-oauth-config\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.583501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.583480    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-serving-cert\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.588845 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.588822    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k87hb\" (UniqueName: \"kubernetes.io/projected/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-kube-api-access-k87hb\") pod \"console-954b68c7f-njw74\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") " pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.689431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.689176    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:26.782700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.782129    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:07:26.786269 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.786194    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9bef378-e519-4b5a-b871-e37044c90eae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slcc9\" (UID: \"c9bef378-e519-4b5a-b871-e37044c90eae\") " pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:07:26.876177 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.876118    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-954b68c7f-njw74"]
Jul 15 04:07:26.881477 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.881443    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk" event={"ID":"a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0","Type":"ContainerStarted","Data":"1c5e6ac805b9f294246f32d7f6ab23a83752e9d3b550bb355aec0bb659db5093"}
Jul 15 04:07:26.882215 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:26.882181    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0489c0c_4a24_4ef8_b63b_38f3a3e3af0f.slice/crio-8145dadf42621732e5494869172a1dc2b1fc0b65f7120f0c25bf386cde368e93 WatchSource:0}: Error finding container 8145dadf42621732e5494869172a1dc2b1fc0b65f7120f0c25bf386cde368e93: Status 404 returned error can't find the container with id 8145dadf42621732e5494869172a1dc2b1fc0b65f7120f0c25bf386cde368e93
Jul 15 04:07:26.899085 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:26.899026    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk" podStartSLOduration=3.513994799 podStartE2EDuration="4.899008152s" totalImagesPullingTime="1.385013353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:26.898980948 +0000 UTC m=+76.210362208" watchObservedRunningTime="2026-07-15 04:07:26.899008152 +0000 UTC m=+76.210389394"
Jul 15 04:07:27.084026 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:27.083942    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tzsqp\""
Jul 15 04:07:27.091915 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:27.091877    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-slcc9"
Jul 15 04:07:27.240422 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:27.240258    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-slcc9"]
Jul 15 04:07:27.243326 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:27.243297    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9bef378_e519_4b5a_b871_e37044c90eae.slice/crio-534aded9086b89ae4b8363a68cd750b18d664daaae77b34d95737a0c8fa6b9de WatchSource:0}: Error finding container 534aded9086b89ae4b8363a68cd750b18d664daaae77b34d95737a0c8fa6b9de: Status 404 returned error can't find the container with id 534aded9086b89ae4b8363a68cd750b18d664daaae77b34d95737a0c8fa6b9de
Jul 15 04:07:27.886781 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:27.886746    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-954b68c7f-njw74" event={"ID":"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f","Type":"ContainerStarted","Data":"8145dadf42621732e5494869172a1dc2b1fc0b65f7120f0c25bf386cde368e93"}
Jul 15 04:07:27.888203 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:27.888170    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-slcc9" event={"ID":"c9bef378-e519-4b5a-b871-e37044c90eae","Type":"ContainerStarted","Data":"534aded9086b89ae4b8363a68cd750b18d664daaae77b34d95737a0c8fa6b9de"}
Jul 15 04:07:27.888439 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:27.888419    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk"
Jul 15 04:07:27.893908 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:27.893887    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ptjnk"
Jul 15 04:07:28.328662 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.328067    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-6mt6f"]
Jul 15 04:07:28.337492 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.337462    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.339948 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.339918    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 15 04:07:28.340726 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.340664    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-6mt6f"]
Jul 15 04:07:28.340975 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.340948    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 04:07:28.341076 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.341005    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 15 04:07:28.341136 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.340953    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-pp57p\""
Jul 15 04:07:28.341380 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.341362    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 04:07:28.341630 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.341548    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 04:07:28.497444 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.497391    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c182da7-587e-40d9-b1db-9594a1ef2569-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.497795 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.497461    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c182da7-587e-40d9-b1db-9594a1ef2569-metrics-client-ca\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.497795 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.497489    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c182da7-587e-40d9-b1db-9594a1ef2569-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.497795 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.497623    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lln6q\" (UniqueName: \"kubernetes.io/projected/3c182da7-587e-40d9-b1db-9594a1ef2569-kube-api-access-lln6q\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.599020 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.598991    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lln6q\" (UniqueName: \"kubernetes.io/projected/3c182da7-587e-40d9-b1db-9594a1ef2569-kube-api-access-lln6q\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.599146 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.599050    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c182da7-587e-40d9-b1db-9594a1ef2569-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.599146 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.599085    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c182da7-587e-40d9-b1db-9594a1ef2569-metrics-client-ca\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.599146 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.599113    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c182da7-587e-40d9-b1db-9594a1ef2569-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.600364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.600266    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c182da7-587e-40d9-b1db-9594a1ef2569-metrics-client-ca\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.602126 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.602103    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c182da7-587e-40d9-b1db-9594a1ef2569-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.603335 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.603299    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c182da7-587e-40d9-b1db-9594a1ef2569-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.607506 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.607473    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lln6q\" (UniqueName: \"kubernetes.io/projected/3c182da7-587e-40d9-b1db-9594a1ef2569-kube-api-access-lln6q\") pod \"prometheus-operator-64785b6675-6mt6f\" (UID: \"3c182da7-587e-40d9-b1db-9594a1ef2569\") " pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.649418 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.649218    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f"
Jul 15 04:07:28.875472 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.875275    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-6mt6f"]
Jul 15 04:07:28.878412 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:28.878383    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c182da7_587e_40d9_b1db_9594a1ef2569.slice/crio-26e8d951fa41e4bfa176e7236eabb886d600dde682cb4b0ccfda3986e906294e WatchSource:0}: Error finding container 26e8d951fa41e4bfa176e7236eabb886d600dde682cb4b0ccfda3986e906294e: Status 404 returned error can't find the container with id 26e8d951fa41e4bfa176e7236eabb886d600dde682cb4b0ccfda3986e906294e
Jul 15 04:07:28.897781 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.897658    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-954b68c7f-njw74" event={"ID":"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f","Type":"ContainerStarted","Data":"d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c"}
Jul 15 04:07:28.901086 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.900934    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f" event={"ID":"3c182da7-587e-40d9-b1db-9594a1ef2569","Type":"ContainerStarted","Data":"26e8d951fa41e4bfa176e7236eabb886d600dde682cb4b0ccfda3986e906294e"}
Jul 15 04:07:28.904257 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.904154    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58499467bd-km6kx" event={"ID":"3579a35b-cc97-4c1e-986d-979b1220e357","Type":"ContainerStarted","Data":"dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8"}
Jul 15 04:07:28.916538 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.915689    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-954b68c7f-njw74" podStartSLOduration=1.228140922 podStartE2EDuration="2.915649134s" totalImagesPullingTime="1.687508212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:07:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:28.91414713 +0000 UTC m=+78.225528376" watchObservedRunningTime="2026-07-15 04:07:28.915649134 +0000 UTC m=+78.227030380"
Jul 15 04:07:28.931788 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:28.930889    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58499467bd-km6kx" podStartSLOduration=1.25549187 podStartE2EDuration="4.930873416s" totalImagesPullingTime="3.675381546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:28.929913975 +0000 UTC m=+78.241295222" watchObservedRunningTime="2026-07-15 04:07:28.930873416 +0000 UTC m=+78.242254669"
Jul 15 04:07:34.705301 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:34.705253    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:34.705764 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:34.705332    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:34.710707 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:34.710684    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:34.932995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:34.932963    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:07:35.884237 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:35.884205    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h7bmp"
Jul 15 04:07:36.689574 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:36.689538    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:36.689791 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:36.689651    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:36.695564 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:36.695534    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:36.939293 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:36.939266    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:07:36.983031 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:36.982957    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58499467bd-km6kx"]
Jul 15 04:07:39.295548 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:39.295508    2524 patch_prober.go:28] interesting pod/image-registry-696df47dcf-x5cn2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 15 04:07:39.296083 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:39.295586    2524 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-696df47dcf-x5cn2" podUID="b925e347-51a5-425f-a836-6333d707ae6d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 04:07:40.952825 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:40.952775    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f" event={"ID":"3c182da7-587e-40d9-b1db-9594a1ef2569","Type":"ContainerStarted","Data":"f97af8d78148a98126b008178daf1f63161c96084383bdd1239a470837074ab6"}
Jul 15 04:07:40.953324 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:40.952898    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f" event={"ID":"3c182da7-587e-40d9-b1db-9594a1ef2569","Type":"ContainerStarted","Data":"c2f60082fad00b5058ca55509ab6b21e0987190b9e606762d5cae8766e4ef681"}
Jul 15 04:07:40.955044 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:40.954681    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-ht4kn" event={"ID":"93c2aa95-043a-4825-b306-f64202da3c9c","Type":"ContainerStarted","Data":"deaf83c4efb4e0d41f3e59eb47c5432d3b92ff1021c130e1a00cc9b275675c94"}
Jul 15 04:07:40.955044 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:40.954853    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-ht4kn"
Jul 15 04:07:40.956808 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:40.956781    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-slcc9" event={"ID":"c9bef378-e519-4b5a-b871-e37044c90eae","Type":"ContainerStarted","Data":"99d24052a2c1100f372e0bc120ccf2624d63ba0651872143d5eb83191ff9730c"}
Jul 15 04:07:40.969346 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:40.969290    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-6mt6f" podStartSLOduration=1.644770441 podStartE2EDuration="12.969271815s" totalImagesPullingTime="11.324501374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:40.967772051 +0000 UTC m=+90.279153298" watchObservedRunningTime="2026-07-15 04:07:40.969271815 +0000 UTC m=+90.280653062"
Jul 15 04:07:40.970428 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:40.970402    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-ht4kn"
Jul 15 04:07:40.983933 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:40.983880    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-ht4kn" podStartSLOduration=3.928424908 podStartE2EDuration="22.983861336s" totalImagesPullingTime="19.055436428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:40.982685797 +0000 UTC m=+90.294067045" watchObservedRunningTime="2026-07-15 04:07:40.983861336 +0000 UTC m=+90.295242648"
Jul 15 04:07:41.965228 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:41.965188    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-slcc9" event={"ID":"c9bef378-e519-4b5a-b871-e37044c90eae","Type":"ContainerStarted","Data":"69a795ac4d9a55d27890351bb722761ee847192f00da95e009803623cb81cfd7"}
Jul 15 04:07:42.672309 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.670825    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-95rx6"]
Jul 15 04:07:42.714618 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.711297    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.714618 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.714105    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 04:07:42.714618 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.714434    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 04:07:42.714869 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.714649    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 04:07:42.714869 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.714847    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5l45x\""
Jul 15 04:07:42.813978 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.813939    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-textfile\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.813978 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.813984    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-wtmp\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.814255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.814033    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-tls\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.814255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.814057    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-accelerators-collector-config\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.814255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.814100    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25949e1b-9c1a-4126-a277-f73daa94bbe7-sys\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.814255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.814123    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25949e1b-9c1a-4126-a277-f73daa94bbe7-metrics-client-ca\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.814255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.814152    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25949e1b-9c1a-4126-a277-f73daa94bbe7-root\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.814255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.814185    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z95q4\" (UniqueName: \"kubernetes.io/projected/25949e1b-9c1a-4126-a277-f73daa94bbe7-kube-api-access-z95q4\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.814255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.814219    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.864898 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.864146    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-696df47dcf-x5cn2"
Jul 15 04:07:42.915256 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.915209    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-tls\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.915431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.915265    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-accelerators-collector-config\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.915431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.915320    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25949e1b-9c1a-4126-a277-f73daa94bbe7-sys\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.915431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.915348    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25949e1b-9c1a-4126-a277-f73daa94bbe7-metrics-client-ca\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.915431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.915378    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25949e1b-9c1a-4126-a277-f73daa94bbe7-root\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.915431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.915409    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z95q4\" (UniqueName: \"kubernetes.io/projected/25949e1b-9c1a-4126-a277-f73daa94bbe7-kube-api-access-z95q4\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.915708 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.915442    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.915708 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.915485    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-textfile\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.915708 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.915509    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-wtmp\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.915848 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.915716    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-wtmp\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.915848 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:42.915832    2524 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 15 04:07:42.915948 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:07:42.915901    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-tls podName:25949e1b-9c1a-4126-a277-f73daa94bbe7 nodeName:}" failed. No retries permitted until 2026-07-15 04:07:43.415879098 +0000 UTC m=+92.727260326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-tls") pod "node-exporter-95rx6" (UID: "25949e1b-9c1a-4126-a277-f73daa94bbe7") : secret "node-exporter-tls" not found
Jul 15 04:07:42.918224 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.916918    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25949e1b-9c1a-4126-a277-f73daa94bbe7-sys\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.918224 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.917450    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25949e1b-9c1a-4126-a277-f73daa94bbe7-metrics-client-ca\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.918224 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.917511    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25949e1b-9c1a-4126-a277-f73daa94bbe7-root\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.918224 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.917775    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-textfile\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.918866 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.918815    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-accelerators-collector-config\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.926495 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.926395    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:42.932630 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:42.929855    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z95q4\" (UniqueName: \"kubernetes.io/projected/25949e1b-9c1a-4126-a277-f73daa94bbe7-kube-api-access-z95q4\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:43.421447 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:43.421389    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-tls\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:43.424321 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:43.424286    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25949e1b-9c1a-4126-a277-f73daa94bbe7-node-exporter-tls\") pod \"node-exporter-95rx6\" (UID: \"25949e1b-9c1a-4126-a277-f73daa94bbe7\") " pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:43.627784 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:43.627558    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-95rx6"
Jul 15 04:07:44.904097 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:44.904059    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25949e1b_9c1a_4126_a277_f73daa94bbe7.slice/crio-6220ab0f4f97b72475f569e2fdacd5a2d8696610c3568fc993428657bb41891d WatchSource:0}: Error finding container 6220ab0f4f97b72475f569e2fdacd5a2d8696610c3568fc993428657bb41891d: Status 404 returned error can't find the container with id 6220ab0f4f97b72475f569e2fdacd5a2d8696610c3568fc993428657bb41891d
Jul 15 04:07:44.985954 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:44.985915    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95rx6" event={"ID":"25949e1b-9c1a-4126-a277-f73daa94bbe7","Type":"ContainerStarted","Data":"6220ab0f4f97b72475f569e2fdacd5a2d8696610c3568fc993428657bb41891d"}
Jul 15 04:07:45.650410 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.648706    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"]
Jul 15 04:07:45.665418 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.665270    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"]
Jul 15 04:07:45.665617 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.665526    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.665617 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.668224    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 15 04:07:45.665617 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.668618    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-trctt\""
Jul 15 04:07:45.665617 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.668806    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 15 04:07:45.665617 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.668976    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 15 04:07:45.665617 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.669221    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 15 04:07:45.665617 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.671278    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c7aqq4mj385a3\""
Jul 15 04:07:45.665617 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.671468    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 15 04:07:45.744562 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.744525    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.744769 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.744568    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.744769 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.744628    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-grpc-tls\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.744769 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.744662    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pz52\" (UniqueName: \"kubernetes.io/projected/603aa743-0412-4183-a7e1-17ceebf776a9-kube-api-access-6pz52\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.744769 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.744709    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.744769 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.744738    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/603aa743-0412-4183-a7e1-17ceebf776a9-metrics-client-ca\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.745023 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.744788    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-tls\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.745023 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.744815    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.845972 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.845929    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-tls\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.846155 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.845982    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.846155 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.846029    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.846155 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.846060    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.846155 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.846096    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-grpc-tls\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.846155 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.846120    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pz52\" (UniqueName: \"kubernetes.io/projected/603aa743-0412-4183-a7e1-17ceebf776a9-kube-api-access-6pz52\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.846155 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.846155    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.846473 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.846177    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/603aa743-0412-4183-a7e1-17ceebf776a9-metrics-client-ca\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.847013 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.846944    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/603aa743-0412-4183-a7e1-17ceebf776a9-metrics-client-ca\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.855181 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.855122    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.855983 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.855941    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.856831 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.856463    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-tls\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.856831 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.856792    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-grpc-tls\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.857145 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.857085    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pz52\" (UniqueName: \"kubernetes.io/projected/603aa743-0412-4183-a7e1-17ceebf776a9-kube-api-access-6pz52\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.857561 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.857516    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.865743 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.865461    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/603aa743-0412-4183-a7e1-17ceebf776a9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7667bdb9fc-4xpzj\" (UID: \"603aa743-0412-4183-a7e1-17ceebf776a9\") " pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.979877 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.979789    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:45.993566 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:45.993524    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-slcc9" event={"ID":"c9bef378-e519-4b5a-b871-e37044c90eae","Type":"ContainerStarted","Data":"996b87d750edc605ff863bed481c715ed08a33b18329d1e702fe292128e3c7f4"}
Jul 15 04:07:46.011901 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:46.011843    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-slcc9" podStartSLOduration=34.217330506 podStartE2EDuration="52.011821321s" totalImagesPullingTime="17.794490815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:06:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:46.010569791 +0000 UTC m=+95.321951038" watchObservedRunningTime="2026-07-15 04:07:46.011821321 +0000 UTC m=+95.323202569"
Jul 15 04:07:46.288839 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:46.288803    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"]
Jul 15 04:07:46.292129 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:46.292101    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603aa743_0412_4183_a7e1_17ceebf776a9.slice/crio-7904012478a148b73c2f1a9e5616072d893e0a72d431646870ff053ecad2e8c8 WatchSource:0}: Error finding container 7904012478a148b73c2f1a9e5616072d893e0a72d431646870ff053ecad2e8c8: Status 404 returned error can't find the container with id 7904012478a148b73c2f1a9e5616072d893e0a72d431646870ff053ecad2e8c8
Jul 15 04:07:47.000191 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.000151    2524 generic.go:356] "Generic (PLEG): container finished" podID="25949e1b-9c1a-4126-a277-f73daa94bbe7" containerID="5698251e52e19d78dba4f762ea617bcf7b79fbf7f389a56b77836422a15def4e" exitCode=0
Jul 15 04:07:47.000647 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.000212    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95rx6" event={"ID":"25949e1b-9c1a-4126-a277-f73daa94bbe7","Type":"ContainerDied","Data":"5698251e52e19d78dba4f762ea617bcf7b79fbf7f389a56b77836422a15def4e"}
Jul 15 04:07:47.002071 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.002035    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj" event={"ID":"603aa743-0412-4183-a7e1-17ceebf776a9","Type":"ContainerStarted","Data":"7904012478a148b73c2f1a9e5616072d893e0a72d431646870ff053ecad2e8c8"}
Jul 15 04:07:47.477579 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.477539    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-855b66cb6-kw6ft"]
Jul 15 04:07:47.499353 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.499320    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-855b66cb6-kw6ft"]
Jul 15 04:07:47.499579 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.499434    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.666982 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.665692    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-trusted-ca-bundle\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.666982 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.665762    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-config\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.666982 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.665801    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-service-ca\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.666982 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.665852    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-serving-cert\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.666982 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.665895    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4gwj\" (UniqueName: \"kubernetes.io/projected/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-kube-api-access-p4gwj\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.666982 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.665925    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-oauth-config\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.666982 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.665955    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-oauth-serving-cert\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.767322 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.767232    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4gwj\" (UniqueName: \"kubernetes.io/projected/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-kube-api-access-p4gwj\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.767322 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.767283    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-oauth-config\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.767322 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.767315    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-oauth-serving-cert\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.767651 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.767362    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-trusted-ca-bundle\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.767651 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.767397    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-config\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.767651 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.767431    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-service-ca\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.767651 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.767477    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-serving-cert\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.768537 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.768502    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-oauth-serving-cert\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.768692 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.768530    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-config\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.768692 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.768557    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-service-ca\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.768692 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.768588    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-trusted-ca-bundle\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.771114 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.771089    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-serving-cert\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.772251 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.772231    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-oauth-config\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.776113 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.776093    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4gwj\" (UniqueName: \"kubernetes.io/projected/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-kube-api-access-p4gwj\") pod \"console-855b66cb6-kw6ft\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") " pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:47.813246 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:47.813205    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:48.010863 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:48.010260    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95rx6" event={"ID":"25949e1b-9c1a-4126-a277-f73daa94bbe7","Type":"ContainerStarted","Data":"02b93bad54fd05a091f6e401ffe76484f4091dafd9795bdec21e3afdc915e384"}
Jul 15 04:07:48.010863 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:48.010327    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95rx6" event={"ID":"25949e1b-9c1a-4126-a277-f73daa94bbe7","Type":"ContainerStarted","Data":"422f2894ef424351d3ce82d8e1bda2debd629844de2f80feaa9da015240e4203"}
Jul 15 04:07:48.021910 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:48.021881    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-855b66cb6-kw6ft"]
Jul 15 04:07:48.024925 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:48.024899    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6560857a_3d00_4b36_a8da_c6c32a8d7ff7.slice/crio-063fec29a51a49fd0009cb2894e528ca8a5c4400bdb543f11b9f5c7948dffb20 WatchSource:0}: Error finding container 063fec29a51a49fd0009cb2894e528ca8a5c4400bdb543f11b9f5c7948dffb20: Status 404 returned error can't find the container with id 063fec29a51a49fd0009cb2894e528ca8a5c4400bdb543f11b9f5c7948dffb20
Jul 15 04:07:48.029843 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:48.029798    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-95rx6" podStartSLOduration=4.269114614 podStartE2EDuration="6.029778772s" totalImagesPullingTime="1.654143706s" totalInitContainerRuntime="106.520452ms" isStatefulPod=true podCreationTimestamp="2026-07-15 04:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:48.029725175 +0000 UTC m=+97.341106423" watchObservedRunningTime="2026-07-15 04:07:48.029778772 +0000 UTC m=+97.341160022"
Jul 15 04:07:49.012983 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.010931    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 04:07:49.039991 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.039952    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855b66cb6-kw6ft" event={"ID":"6560857a-3d00-4b36-a8da-c6c32a8d7ff7","Type":"ContainerStarted","Data":"ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5"}
Jul 15 04:07:49.040168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.040011    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 04:07:49.040168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.040082    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.040168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.040105    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855b66cb6-kw6ft" event={"ID":"6560857a-3d00-4b36-a8da-c6c32a8d7ff7","Type":"ContainerStarted","Data":"063fec29a51a49fd0009cb2894e528ca8a5c4400bdb543f11b9f5c7948dffb20"}
Jul 15 04:07:49.044364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.041909    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-855b66cb6-kw6ft" podStartSLOduration=2.041891601 podStartE2EDuration="2.041891601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:07:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:49.04151916 +0000 UTC m=+98.352900408" watchObservedRunningTime="2026-07-15 04:07:49.041891601 +0000 UTC m=+98.353272850"
Jul 15 04:07:49.044364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.042763    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 04:07:49.044364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.043018    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 04:07:49.044364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.043212    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 04:07:49.044364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.043688    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 04:07:49.044364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.043810    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 04:07:49.044364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.043931    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-biif1g6dsc7qp\""
Jul 15 04:07:49.044364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.044032    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 04:07:49.044364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.044161    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 04:07:49.044364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.044237    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 04:07:49.044980 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.044562    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 04:07:49.044980 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.044853    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 04:07:49.045085 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.045031    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 04:07:49.045150 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.045132    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 04:07:49.046642 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.045668    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5lql4\""
Jul 15 04:07:49.047110 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.046951    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 04:07:49.047110 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.047021    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 04:07:49.064304 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.064275    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 04:07:49.184552 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.184511    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.184755 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.184562    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.184755 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.184654    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.184755 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.184703    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.184928 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.184777    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.184928 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.184834    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185165    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185379 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185299    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185379 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185323    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185505 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185378    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185505 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185407    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185505 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185429    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185505 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185481    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185505 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185497    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185808 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185522    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185808 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185571    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185808 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185588    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx5b8\" (UniqueName: \"kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-kube-api-access-nx5b8\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185808 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185673    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-config\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185808 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185704    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-config-out\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.185808 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.185740    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-web-config\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.286653    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.286720    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.286753    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.286791    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.286831    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.286863    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.286900    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.286928    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.286952    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.286995    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.287019    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.287046    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.287092    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.287121    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx5b8\" (UniqueName: \"kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-kube-api-access-nx5b8\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.287149    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-config\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.287173    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-config-out\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.287450 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.287196    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-web-config\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.288430 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.287229    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.288430 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.287255    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.288430 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.287283    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.288430 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.287841    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.288430 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.288062    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.288430 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.288347    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.290927 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.290659    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.291503 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.291196    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-config-out\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.291841 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.291813    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.292972 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.292915    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.293506 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.293484    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.294539 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.294128    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.294726 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.294705    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.299794 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.295530    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.299794 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.295871    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.299794 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.296032    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.299794 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.296437    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.299794 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.296943    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-config\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.299794 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.297514    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.299794 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.298004    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-web-config\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.299794 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.298978    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.299794 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.299076    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.304692 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.304667    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx5b8\" (UniqueName: \"kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-kube-api-access-nx5b8\") pod \"prometheus-k8s-0\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.371245 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.371203    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:07:49.559744 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:49.559719    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 04:07:49.628569 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:07:49.628526    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768ff916_ac83_4ead_9382_cadd8c57e0ba.slice/crio-1ba4790de997c71130745e0ee3a94cb8f3073ae52025ff0169f7ee1a02c640e1 WatchSource:0}: Error finding container 1ba4790de997c71130745e0ee3a94cb8f3073ae52025ff0169f7ee1a02c640e1: Status 404 returned error can't find the container with id 1ba4790de997c71130745e0ee3a94cb8f3073ae52025ff0169f7ee1a02c640e1
Jul 15 04:07:50.025437 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:50.025399    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerStarted","Data":"1ba4790de997c71130745e0ee3a94cb8f3073ae52025ff0169f7ee1a02c640e1"}
Jul 15 04:07:50.028446 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:50.028417    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj" event={"ID":"603aa743-0412-4183-a7e1-17ceebf776a9","Type":"ContainerStarted","Data":"02359978e94b82c3afc494730198fe7f7c1b67a5cf5c1602278accea1a00321b"}
Jul 15 04:07:50.028575 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:50.028511    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj" event={"ID":"603aa743-0412-4183-a7e1-17ceebf776a9","Type":"ContainerStarted","Data":"1dc471a0bafab42ff817150ee7f0af1bd9c233466eee6f39625cbd439a9afa91"}
Jul 15 04:07:50.028575 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:50.028543    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj" event={"ID":"603aa743-0412-4183-a7e1-17ceebf776a9","Type":"ContainerStarted","Data":"eea7ed4a4598429aa2e482f8ccceb5097c662635aea1b560bdc3ad8fdc0bb17d"}
Jul 15 04:07:52.040466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:52.040363    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj" event={"ID":"603aa743-0412-4183-a7e1-17ceebf776a9","Type":"ContainerStarted","Data":"9875d37ae113d37516b0154fccab0ca770feea6c59692f7e57d0ba5f42e59e42"}
Jul 15 04:07:52.040940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:52.040489    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj" event={"ID":"603aa743-0412-4183-a7e1-17ceebf776a9","Type":"ContainerStarted","Data":"0e4435d153b32cc74f1c3444e7b547dd4b4eb67b6f928dc1a675e0815a8f2ff3"}
Jul 15 04:07:52.040940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:52.040529    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj" event={"ID":"603aa743-0412-4183-a7e1-17ceebf776a9","Type":"ContainerStarted","Data":"f53673000e84bbbb5e7e09dfd2854e76878bc65368fd690b28b823ec2525eea7"}
Jul 15 04:07:52.041659 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:52.041637    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:52.043291 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:52.043267    2524 generic.go:356] "Generic (PLEG): container finished" podID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerID="7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8" exitCode=0
Jul 15 04:07:52.043410 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:52.043323    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerDied","Data":"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"}
Jul 15 04:07:52.064919 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:52.064870    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj" podStartSLOduration=2.285725316 podStartE2EDuration="7.064852992s" totalImagesPullingTime="4.779127676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:07:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:52.063408638 +0000 UTC m=+101.374789896" watchObservedRunningTime="2026-07-15 04:07:52.064852992 +0000 UTC m=+101.376234239"
Jul 15 04:07:53.859162 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:53.859134    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-txk5x"
Jul 15 04:07:54.062949 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:54.062915    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7667bdb9fc-4xpzj"
Jul 15 04:07:57.076865 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:57.076825    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerStarted","Data":"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"}
Jul 15 04:07:57.077292 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:57.076908    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerStarted","Data":"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"}
Jul 15 04:07:57.077292 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:57.076949    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerStarted","Data":"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"}
Jul 15 04:07:57.077292 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:57.076987    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerStarted","Data":"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"}
Jul 15 04:07:57.077292 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:57.077025    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerStarted","Data":"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"}
Jul 15 04:07:57.077292 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:57.077060    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerStarted","Data":"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"}
Jul 15 04:07:57.105709 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:57.105623    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.260114708 podStartE2EDuration="9.105570339s" totalImagesPullingTime="5.821785424s" totalInitContainerRuntime="23.670207ms" isStatefulPod=false podCreationTimestamp="2026-07-15 04:07:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:07:57.104357758 +0000 UTC m=+106.415739039" watchObservedRunningTime="2026-07-15 04:07:57.105570339 +0000 UTC m=+106.416951588"
Jul 15 04:07:57.813674 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:57.813633    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:57.813926 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:57.813707    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:57.818644 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:57.818618    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:58.085482 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:58.085405    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:07:58.131274 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:58.131237    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-954b68c7f-njw74"]
Jul 15 04:07:59.372081 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:07:59.372047    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:08:02.010674 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.010583    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58499467bd-km6kx" podUID="3579a35b-cc97-4c1e-986d-979b1220e357" containerName="console" containerID="cri-o://dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8" gracePeriod=15
Jul 15 04:08:02.282910 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.282842    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:08:02.296838 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.296811    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-console-config\") pod \"3579a35b-cc97-4c1e-986d-979b1220e357\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") "
Jul 15 04:08:02.296994 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.296868    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5fng\" (UniqueName: \"kubernetes.io/projected/3579a35b-cc97-4c1e-986d-979b1220e357-kube-api-access-s5fng\") pod \"3579a35b-cc97-4c1e-986d-979b1220e357\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") "
Jul 15 04:08:02.296994 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.296909    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-service-ca\") pod \"3579a35b-cc97-4c1e-986d-979b1220e357\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") "
Jul 15 04:08:02.296994 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.296937    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-oauth-serving-cert\") pod \"3579a35b-cc97-4c1e-986d-979b1220e357\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") "
Jul 15 04:08:02.296994 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.296988    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-oauth-config\") pod \"3579a35b-cc97-4c1e-986d-979b1220e357\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") "
Jul 15 04:08:02.297192 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.297025    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-serving-cert\") pod \"3579a35b-cc97-4c1e-986d-979b1220e357\" (UID: \"3579a35b-cc97-4c1e-986d-979b1220e357\") "
Jul 15 04:08:02.297365 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.297332    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-console-config" (OuterVolumeSpecName: "console-config") pod "3579a35b-cc97-4c1e-986d-979b1220e357" (UID: "3579a35b-cc97-4c1e-986d-979b1220e357"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:08:02.297365 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.297356    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3579a35b-cc97-4c1e-986d-979b1220e357" (UID: "3579a35b-cc97-4c1e-986d-979b1220e357"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:08:02.297693 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.297666    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-service-ca" (OuterVolumeSpecName: "service-ca") pod "3579a35b-cc97-4c1e-986d-979b1220e357" (UID: "3579a35b-cc97-4c1e-986d-979b1220e357"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:08:02.299340 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.299311    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3579a35b-cc97-4c1e-986d-979b1220e357-kube-api-access-s5fng" (OuterVolumeSpecName: "kube-api-access-s5fng") pod "3579a35b-cc97-4c1e-986d-979b1220e357" (UID: "3579a35b-cc97-4c1e-986d-979b1220e357"). InnerVolumeSpecName "kube-api-access-s5fng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:08:02.299807 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.299777    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3579a35b-cc97-4c1e-986d-979b1220e357" (UID: "3579a35b-cc97-4c1e-986d-979b1220e357"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:08:02.299807 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.299790    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3579a35b-cc97-4c1e-986d-979b1220e357" (UID: "3579a35b-cc97-4c1e-986d-979b1220e357"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:08:02.397650 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.397613    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5fng\" (UniqueName: \"kubernetes.io/projected/3579a35b-cc97-4c1e-986d-979b1220e357-kube-api-access-s5fng\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:02.397650 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.397644    2524 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:02.397650 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.397658    2524 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-oauth-serving-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:02.397981 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.397672    2524 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-oauth-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:02.397981 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.397687    2524 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3579a35b-cc97-4c1e-986d-979b1220e357-console-serving-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:02.397981 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:02.397699    2524 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3579a35b-cc97-4c1e-986d-979b1220e357-console-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:03.107315 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:03.107282    2524 generic.go:356] "Generic (PLEG): container finished" podID="3579a35b-cc97-4c1e-986d-979b1220e357" containerID="dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8" exitCode=0
Jul 15 04:08:03.107854 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:03.107348    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58499467bd-km6kx"
Jul 15 04:08:03.107854 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:03.107356    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58499467bd-km6kx" event={"ID":"3579a35b-cc97-4c1e-986d-979b1220e357","Type":"ContainerDied","Data":"dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8"}
Jul 15 04:08:03.107854 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:03.107420    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58499467bd-km6kx" event={"ID":"3579a35b-cc97-4c1e-986d-979b1220e357","Type":"ContainerDied","Data":"3fd417a89d98e524bd5b85221338133bbca0c2546d889dffbba1602f88b9e4e0"}
Jul 15 04:08:03.107854 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:03.107444    2524 scope.go:117] "RemoveContainer" containerID="dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8"
Jul 15 04:08:03.116351 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:03.116332    2524 scope.go:117] "RemoveContainer" containerID="dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8"
Jul 15 04:08:03.116657 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:08:03.116634    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8\": container with ID starting with dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8 not found: ID does not exist" containerID="dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8"
Jul 15 04:08:03.116707 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:03.116666    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8"} err="failed to get container status \"dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8\": rpc error: code = NotFound desc = could not find container \"dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8\": container with ID starting with dcd8fb8042f5dbe9b2f93dbc88cfebffb291db972b0207f3b15d1a04e7dc1bf8 not found: ID does not exist"
Jul 15 04:08:03.128709 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:03.128683    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58499467bd-km6kx"]
Jul 15 04:08:03.133398 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:03.131855    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58499467bd-km6kx"]
Jul 15 04:08:03.431907 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:03.431874    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3579a35b-cc97-4c1e-986d-979b1220e357" path="/var/lib/kubelet/pods/3579a35b-cc97-4c1e-986d-979b1220e357/volumes"
Jul 15 04:08:15.428234 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:15.428184    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-44srg_52b6b8d4-5eee-4b5d-96cb-befdb1d9b901/serve-healthcheck-canary/0.log"
Jul 15 04:08:23.151932 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.151892    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-954b68c7f-njw74" podUID="e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" containerName="console" containerID="cri-o://d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c" gracePeriod=15
Jul 15 04:08:23.427756 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.427691    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:08:23.471479 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.471449    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-oauth-config\") pod \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") "
Jul 15 04:08:23.471479 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.471485    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-serving-cert\") pod \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") "
Jul 15 04:08:23.471700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.471504    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k87hb\" (UniqueName: \"kubernetes.io/projected/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-kube-api-access-k87hb\") pod \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") "
Jul 15 04:08:23.471700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.471551    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-oauth-serving-cert\") pod \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") "
Jul 15 04:08:23.471700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.471578    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-service-ca\") pod \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") "
Jul 15 04:08:23.471700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.471628    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-trusted-ca-bundle\") pod \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") "
Jul 15 04:08:23.471700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.471673    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-config\") pod \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\" (UID: \"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f\") "
Jul 15 04:08:23.472115 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.472084    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" (UID: "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:08:23.472115 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.472094    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-service-ca" (OuterVolumeSpecName: "service-ca") pod "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" (UID: "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:08:23.472244 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.472098    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" (UID: "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:08:23.472244 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.472132    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-config" (OuterVolumeSpecName: "console-config") pod "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" (UID: "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:08:23.473788 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.473749    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" (UID: "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:08:23.474282 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.474256    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" (UID: "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:08:23.474282 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.474269    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-kube-api-access-k87hb" (OuterVolumeSpecName: "kube-api-access-k87hb") pod "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" (UID: "e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f"). InnerVolumeSpecName "kube-api-access-k87hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:08:23.572525 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.572487    2524 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:23.572525 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.572519    2524 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-trusted-ca-bundle\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:23.572525 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.572528    2524 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:23.572525 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.572537    2524 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-oauth-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:23.572804 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.572547    2524 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-console-serving-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:23.572804 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.572556    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k87hb\" (UniqueName: \"kubernetes.io/projected/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-kube-api-access-k87hb\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:23.572804 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:23.572565    2524 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f-oauth-serving-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:08:24.187945 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:24.187909    2524 generic.go:356] "Generic (PLEG): container finished" podID="e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" containerID="d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c" exitCode=0
Jul 15 04:08:24.188367 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:24.187961    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-954b68c7f-njw74"
Jul 15 04:08:24.188367 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:24.188002    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-954b68c7f-njw74" event={"ID":"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f","Type":"ContainerDied","Data":"d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c"}
Jul 15 04:08:24.188367 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:24.188053    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-954b68c7f-njw74" event={"ID":"e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f","Type":"ContainerDied","Data":"8145dadf42621732e5494869172a1dc2b1fc0b65f7120f0c25bf386cde368e93"}
Jul 15 04:08:24.188367 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:24.188074    2524 scope.go:117] "RemoveContainer" containerID="d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c"
Jul 15 04:08:24.196468 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:24.196452    2524 scope.go:117] "RemoveContainer" containerID="d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c"
Jul 15 04:08:24.196720 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:08:24.196701    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c\": container with ID starting with d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c not found: ID does not exist" containerID="d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c"
Jul 15 04:08:24.196786 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:24.196728    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c"} err="failed to get container status \"d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c\": rpc error: code = NotFound desc = could not find container \"d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c\": container with ID starting with d25751296ca377c3719ae4b9d582831ebe8e72563f3d09cc2c63df27f243468c not found: ID does not exist"
Jul 15 04:08:24.208995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:24.208969    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-954b68c7f-njw74"]
Jul 15 04:08:24.212750 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:24.212732    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-954b68c7f-njw74"]
Jul 15 04:08:25.431614 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:25.431570    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" path="/var/lib/kubelet/pods/e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f/volumes"
Jul 15 04:08:49.371975 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:49.371929    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:08:49.391934 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:49.391909    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:08:50.309952 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:08:50.309923    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:07.324686 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.324449    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 04:09:07.325423 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.325383    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="prometheus" containerID="cri-o://2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc" gracePeriod=600
Jul 15 04:09:07.325646 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.325391    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy-thanos" containerID="cri-o://9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d" gracePeriod=600
Jul 15 04:09:07.325770 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.325394    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="thanos-sidecar" containerID="cri-o://faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af" gracePeriod=600
Jul 15 04:09:07.325770 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.325394    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy" containerID="cri-o://19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483" gracePeriod=600
Jul 15 04:09:07.325867 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.325394    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="config-reloader" containerID="cri-o://c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1" gracePeriod=600
Jul 15 04:09:07.325913 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.325454    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy-web" containerID="cri-o://7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006" gracePeriod=600
Jul 15 04:09:07.586180 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.585858    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:07.653357 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653320    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-config-out\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653543 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653378    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-grpc-tls\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653543 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653410    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-db\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653543 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653436    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-metrics-client-certs\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653543 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653462    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-tls-assets\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653543 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653493    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-1\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653543 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653518    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-web-config\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653566    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-thanos-prometheus-http-client-file\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653609    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-0\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653641    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-metrics-client-ca\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653691    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-trusted-ca-bundle\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653725    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-serving-certs-ca-bundle\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653751    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-kubelet-serving-ca-bundle\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653774    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-config\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653809    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.653904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653872    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.654501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653913    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-2\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.654501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653938    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-tls\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.654501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.653968    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx5b8\" (UniqueName: \"kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-kube-api-access-nx5b8\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.654501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.654001    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-kube-rbac-proxy\") pod \"768ff916-ac83-4ead-9382-cadd8c57e0ba\" (UID: \"768ff916-ac83-4ead-9382-cadd8c57e0ba\") "
Jul 15 04:09:07.654501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.654488    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:09:07.655038 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.654803    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:09:07.655038 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.654813    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:09:07.655186 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.655129    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:09:07.655444 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.655410    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:09:07.655582 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.655539    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:09:07.656888 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.656660    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:09:07.656888 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.656762    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:09:07.657264 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.657238    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:09:07.657891 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.657862    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:09:07.658921 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.658898    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-config" (OuterVolumeSpecName: "config") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:09:07.659328 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.659285    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:09:07.659530 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.659503    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:09:07.659748 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.659729    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:09:07.659861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.659842    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:09:07.659861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.659851    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-config-out" (OuterVolumeSpecName: "config-out") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:09:07.659994 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.659958    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:09:07.660754 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.660724    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:09:07.661724 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.661703    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-kube-api-access-nx5b8" (OuterVolumeSpecName: "kube-api-access-nx5b8") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "kube-api-access-nx5b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:09:07.671002 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.670969    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-web-config" (OuterVolumeSpecName: "web-config") pod "768ff916-ac83-4ead-9382-cadd8c57e0ba" (UID: "768ff916-ac83-4ead-9382-cadd8c57e0ba"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:09:07.754733 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754690    2524 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754733 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754722    2524 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754733 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754732    2524 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754733 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754742    2524 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754750    2524 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754760    2524 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754769    2524 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754779    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx5b8\" (UniqueName: \"kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-kube-api-access-nx5b8\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754787    2524 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-kube-rbac-proxy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754796    2524 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-config-out\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754804    2524 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-grpc-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754811    2524 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-db\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754819    2524 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-secret-metrics-client-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754827    2524 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/768ff916-ac83-4ead-9382-cadd8c57e0ba-tls-assets\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754837    2524 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754845    2524 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-web-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754853    2524 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/768ff916-ac83-4ead-9382-cadd8c57e0ba-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754860    2524 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754870    2524 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-configmap-metrics-client-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:07.754995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:07.754878    2524 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/768ff916-ac83-4ead-9382-cadd8c57e0ba-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:08.374334 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374301    2524 generic.go:356] "Generic (PLEG): container finished" podID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerID="9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d" exitCode=0
Jul 15 04:09:08.374334 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374330    2524 generic.go:356] "Generic (PLEG): container finished" podID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerID="19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483" exitCode=0
Jul 15 04:09:08.374334 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374336    2524 generic.go:356] "Generic (PLEG): container finished" podID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerID="7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006" exitCode=0
Jul 15 04:09:08.374334 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374342    2524 generic.go:356] "Generic (PLEG): container finished" podID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerID="faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af" exitCode=0
Jul 15 04:09:08.374861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374349    2524 generic.go:356] "Generic (PLEG): container finished" podID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerID="c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1" exitCode=0
Jul 15 04:09:08.374861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374354    2524 generic.go:356] "Generic (PLEG): container finished" podID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerID="2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc" exitCode=0
Jul 15 04:09:08.374861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374375    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.374861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374391    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerDied","Data":"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"}
Jul 15 04:09:08.374861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374473    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerDied","Data":"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"}
Jul 15 04:09:08.374861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374502    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerDied","Data":"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"}
Jul 15 04:09:08.374861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374537    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerDied","Data":"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"}
Jul 15 04:09:08.374861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374542    2524 scope.go:117] "RemoveContainer" containerID="9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"
Jul 15 04:09:08.374861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374573    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerDied","Data":"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"}
Jul 15 04:09:08.374861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374628    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerDied","Data":"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"}
Jul 15 04:09:08.374861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.374670    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"768ff916-ac83-4ead-9382-cadd8c57e0ba","Type":"ContainerDied","Data":"1ba4790de997c71130745e0ee3a94cb8f3073ae52025ff0169f7ee1a02c640e1"}
Jul 15 04:09:08.383227 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.383195    2524 scope.go:117] "RemoveContainer" containerID="19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"
Jul 15 04:09:08.390422 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.390403    2524 scope.go:117] "RemoveContainer" containerID="7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"
Jul 15 04:09:08.396987 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.396955    2524 scope.go:117] "RemoveContainer" containerID="faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"
Jul 15 04:09:08.398012 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.397937    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 04:09:08.403101 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.403078    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 04:09:08.404793 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.404778    2524 scope.go:117] "RemoveContainer" containerID="c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"
Jul 15 04:09:08.411324 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.411306    2524 scope.go:117] "RemoveContainer" containerID="2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"
Jul 15 04:09:08.418437 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.418419    2524 scope.go:117] "RemoveContainer" containerID="7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"
Jul 15 04:09:08.424396 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424376    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 04:09:08.424664 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424651    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3579a35b-cc97-4c1e-986d-979b1220e357" containerName="console"
Jul 15 04:09:08.424715 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424666    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="3579a35b-cc97-4c1e-986d-979b1220e357" containerName="console"
Jul 15 04:09:08.424715 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424676    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy-web"
Jul 15 04:09:08.424715 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424681    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy-web"
Jul 15 04:09:08.424715 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424693    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="init-config-reloader"
Jul 15 04:09:08.424715 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424698    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="init-config-reloader"
Jul 15 04:09:08.424715 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424705    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="thanos-sidecar"
Jul 15 04:09:08.424715 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424710    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="thanos-sidecar"
Jul 15 04:09:08.424715 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424717    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" containerName="console"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424722    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" containerName="console"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424734    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="prometheus"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424738    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="prometheus"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424744    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424749    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424756    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy-thanos"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424760    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy-thanos"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424768    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="config-reloader"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424773    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="config-reloader"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424815    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="thanos-sidecar"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424822    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="3579a35b-cc97-4c1e-986d-979b1220e357" containerName="console"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424828    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0489c0c-4a24-4ef8-b63b-38f3a3e3af0f" containerName="console"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424833    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="prometheus"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424838    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy-web"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424843    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="config-reloader"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424850    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy"
Jul 15 04:09:08.424942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.424856    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" containerName="kube-rbac-proxy-thanos"
Jul 15 04:09:08.425421 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.425362    2524 scope.go:117] "RemoveContainer" containerID="9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"
Jul 15 04:09:08.425673 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:09:08.425651    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": container with ID starting with 9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d not found: ID does not exist" containerID="9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"
Jul 15 04:09:08.425720 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.425688    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"} err="failed to get container status \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": rpc error: code = NotFound desc = could not find container \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": container with ID starting with 9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d not found: ID does not exist"
Jul 15 04:09:08.425720 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.425710    2524 scope.go:117] "RemoveContainer" containerID="19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"
Jul 15 04:09:08.425981 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:09:08.425962    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": container with ID starting with 19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483 not found: ID does not exist" containerID="19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"
Jul 15 04:09:08.426066 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.425986    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"} err="failed to get container status \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": rpc error: code = NotFound desc = could not find container \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": container with ID starting with 19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483 not found: ID does not exist"
Jul 15 04:09:08.426066 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.426004    2524 scope.go:117] "RemoveContainer" containerID="7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"
Jul 15 04:09:08.426251 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:09:08.426234    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": container with ID starting with 7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006 not found: ID does not exist" containerID="7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"
Jul 15 04:09:08.426286 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.426257    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"} err="failed to get container status \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": rpc error: code = NotFound desc = could not find container \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": container with ID starting with 7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006 not found: ID does not exist"
Jul 15 04:09:08.426286 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.426272    2524 scope.go:117] "RemoveContainer" containerID="faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"
Jul 15 04:09:08.426552 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:09:08.426488    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": container with ID starting with faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af not found: ID does not exist" containerID="faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"
Jul 15 04:09:08.426552 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.426510    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"} err="failed to get container status \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": rpc error: code = NotFound desc = could not find container \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": container with ID starting with faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af not found: ID does not exist"
Jul 15 04:09:08.426552 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.426523    2524 scope.go:117] "RemoveContainer" containerID="c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"
Jul 15 04:09:08.426844 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:09:08.426790    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": container with ID starting with c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1 not found: ID does not exist" containerID="c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"
Jul 15 04:09:08.426844 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.426816    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"} err="failed to get container status \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": rpc error: code = NotFound desc = could not find container \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": container with ID starting with c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1 not found: ID does not exist"
Jul 15 04:09:08.426940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.426849    2524 scope.go:117] "RemoveContainer" containerID="2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"
Jul 15 04:09:08.427119 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:09:08.427104    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": container with ID starting with 2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc not found: ID does not exist" containerID="2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"
Jul 15 04:09:08.427171 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.427125    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"} err="failed to get container status \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": rpc error: code = NotFound desc = could not find container \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": container with ID starting with 2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc not found: ID does not exist"
Jul 15 04:09:08.427171 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.427139    2524 scope.go:117] "RemoveContainer" containerID="7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"
Jul 15 04:09:08.427371 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:09:08.427355    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": container with ID starting with 7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8 not found: ID does not exist" containerID="7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"
Jul 15 04:09:08.427410 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.427375    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"} err="failed to get container status \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": rpc error: code = NotFound desc = could not find container \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": container with ID starting with 7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8 not found: ID does not exist"
Jul 15 04:09:08.427410 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.427389    2524 scope.go:117] "RemoveContainer" containerID="9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"
Jul 15 04:09:08.427634 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.427617    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"} err="failed to get container status \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": rpc error: code = NotFound desc = could not find container \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": container with ID starting with 9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d not found: ID does not exist"
Jul 15 04:09:08.427634 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.427633    2524 scope.go:117] "RemoveContainer" containerID="19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"
Jul 15 04:09:08.427860 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.427839    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"} err="failed to get container status \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": rpc error: code = NotFound desc = could not find container \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": container with ID starting with 19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483 not found: ID does not exist"
Jul 15 04:09:08.427929 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.427861    2524 scope.go:117] "RemoveContainer" containerID="7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"
Jul 15 04:09:08.428109 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.428088    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"} err="failed to get container status \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": rpc error: code = NotFound desc = could not find container \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": container with ID starting with 7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006 not found: ID does not exist"
Jul 15 04:09:08.428109 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.428108    2524 scope.go:117] "RemoveContainer" containerID="faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"
Jul 15 04:09:08.428339 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.428321    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"} err="failed to get container status \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": rpc error: code = NotFound desc = could not find container \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": container with ID starting with faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af not found: ID does not exist"
Jul 15 04:09:08.428382 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.428340    2524 scope.go:117] "RemoveContainer" containerID="c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"
Jul 15 04:09:08.428543 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.428528    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"} err="failed to get container status \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": rpc error: code = NotFound desc = could not find container \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": container with ID starting with c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1 not found: ID does not exist"
Jul 15 04:09:08.428587 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.428543    2524 scope.go:117] "RemoveContainer" containerID="2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"
Jul 15 04:09:08.428764 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.428746    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"} err="failed to get container status \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": rpc error: code = NotFound desc = could not find container \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": container with ID starting with 2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc not found: ID does not exist"
Jul 15 04:09:08.428837 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.428767    2524 scope.go:117] "RemoveContainer" containerID="7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"
Jul 15 04:09:08.429089 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.429069    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.429239 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.429168    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"} err="failed to get container status \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": rpc error: code = NotFound desc = could not find container \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": container with ID starting with 7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8 not found: ID does not exist"
Jul 15 04:09:08.429239 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.429193    2524 scope.go:117] "RemoveContainer" containerID="9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"
Jul 15 04:09:08.429855 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.429816    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"} err="failed to get container status \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": rpc error: code = NotFound desc = could not find container \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": container with ID starting with 9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d not found: ID does not exist"
Jul 15 04:09:08.429971 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.429856    2524 scope.go:117] "RemoveContainer" containerID="19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"
Jul 15 04:09:08.430144 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.430122    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"} err="failed to get container status \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": rpc error: code = NotFound desc = could not find container \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": container with ID starting with 19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483 not found: ID does not exist"
Jul 15 04:09:08.430192 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.430146    2524 scope.go:117] "RemoveContainer" containerID="7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"
Jul 15 04:09:08.430410 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.430385    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"} err="failed to get container status \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": rpc error: code = NotFound desc = could not find container \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": container with ID starting with 7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006 not found: ID does not exist"
Jul 15 04:09:08.430461 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.430413    2524 scope.go:117] "RemoveContainer" containerID="faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"
Jul 15 04:09:08.430648 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.430630    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"} err="failed to get container status \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": rpc error: code = NotFound desc = could not find container \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": container with ID starting with faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af not found: ID does not exist"
Jul 15 04:09:08.430759 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.430650    2524 scope.go:117] "RemoveContainer" containerID="c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"
Jul 15 04:09:08.430901 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.430884    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"} err="failed to get container status \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": rpc error: code = NotFound desc = could not find container \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": container with ID starting with c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1 not found: ID does not exist"
Jul 15 04:09:08.430948 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.430902    2524 scope.go:117] "RemoveContainer" containerID="2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"
Jul 15 04:09:08.431104 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.431087    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"} err="failed to get container status \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": rpc error: code = NotFound desc = could not find container \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": container with ID starting with 2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc not found: ID does not exist"
Jul 15 04:09:08.431162 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.431105    2524 scope.go:117] "RemoveContainer" containerID="7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"
Jul 15 04:09:08.431333 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.431316    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"} err="failed to get container status \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": rpc error: code = NotFound desc = could not find container \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": container with ID starting with 7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8 not found: ID does not exist"
Jul 15 04:09:08.431378 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.431334    2524 scope.go:117] "RemoveContainer" containerID="9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"
Jul 15 04:09:08.431567 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.431547    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"} err="failed to get container status \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": rpc error: code = NotFound desc = could not find container \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": container with ID starting with 9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d not found: ID does not exist"
Jul 15 04:09:08.431656 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.431568    2524 scope.go:117] "RemoveContainer" containerID="19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"
Jul 15 04:09:08.431807 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.431787    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"} err="failed to get container status \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": rpc error: code = NotFound desc = could not find container \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": container with ID starting with 19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483 not found: ID does not exist"
Jul 15 04:09:08.431885 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.431809    2524 scope.go:117] "RemoveContainer" containerID="7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"
Jul 15 04:09:08.432041 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432021    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"} err="failed to get container status \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": rpc error: code = NotFound desc = could not find container \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": container with ID starting with 7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006 not found: ID does not exist"
Jul 15 04:09:08.432107 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432043    2524 scope.go:117] "RemoveContainer" containerID="faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"
Jul 15 04:09:08.432258 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432231    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"} err="failed to get container status \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": rpc error: code = NotFound desc = could not find container \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": container with ID starting with faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af not found: ID does not exist"
Jul 15 04:09:08.432258 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432251    2524 scope.go:117] "RemoveContainer" containerID="c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432492    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432530    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432554    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"} err="failed to get container status \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": rpc error: code = NotFound desc = could not find container \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": container with ID starting with c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1 not found: ID does not exist"
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432566    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432574    2524 scope.go:117] "RemoveContainer" containerID="2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432566    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432735    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432803    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432837    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432747    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432942    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432947    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432968    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 04:09:08.433080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.432924    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 04:09:08.433909 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.433095    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 04:09:08.433909 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.433140    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-biif1g6dsc7qp\""
Jul 15 04:09:08.433909 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.433336    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5lql4\""
Jul 15 04:09:08.433909 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.433505    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"} err="failed to get container status \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": rpc error: code = NotFound desc = could not find container \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": container with ID starting with 2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc not found: ID does not exist"
Jul 15 04:09:08.433909 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.433529    2524 scope.go:117] "RemoveContainer" containerID="7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"
Jul 15 04:09:08.433909 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.433786    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"} err="failed to get container status \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": rpc error: code = NotFound desc = could not find container \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": container with ID starting with 7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8 not found: ID does not exist"
Jul 15 04:09:08.433909 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.433810    2524 scope.go:117] "RemoveContainer" containerID="9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"
Jul 15 04:09:08.434649 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.434589    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"} err="failed to get container status \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": rpc error: code = NotFound desc = could not find container \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": container with ID starting with 9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d not found: ID does not exist"
Jul 15 04:09:08.434754 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.434652    2524 scope.go:117] "RemoveContainer" containerID="19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"
Jul 15 04:09:08.439844 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.439814    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"} err="failed to get container status \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": rpc error: code = NotFound desc = could not find container \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": container with ID starting with 19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483 not found: ID does not exist"
Jul 15 04:09:08.439951 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.439852    2524 scope.go:117] "RemoveContainer" containerID="7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"
Jul 15 04:09:08.440812 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.440776    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"} err="failed to get container status \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": rpc error: code = NotFound desc = could not find container \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": container with ID starting with 7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006 not found: ID does not exist"
Jul 15 04:09:08.440812 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.440808    2524 scope.go:117] "RemoveContainer" containerID="faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"
Jul 15 04:09:08.441095 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.441069    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"} err="failed to get container status \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": rpc error: code = NotFound desc = could not find container \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": container with ID starting with faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af not found: ID does not exist"
Jul 15 04:09:08.441154 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.441099    2524 scope.go:117] "RemoveContainer" containerID="c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"
Jul 15 04:09:08.441363 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.441340    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"} err="failed to get container status \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": rpc error: code = NotFound desc = could not find container \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": container with ID starting with c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1 not found: ID does not exist"
Jul 15 04:09:08.441438 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.441363    2524 scope.go:117] "RemoveContainer" containerID="2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"
Jul 15 04:09:08.441485 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.441457    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 04:09:08.441634 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.441617    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"} err="failed to get container status \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": rpc error: code = NotFound desc = could not find container \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": container with ID starting with 2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc not found: ID does not exist"
Jul 15 04:09:08.441681 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.441635    2524 scope.go:117] "RemoveContainer" containerID="7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"
Jul 15 04:09:08.441911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.441888    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"} err="failed to get container status \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": rpc error: code = NotFound desc = could not find container \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": container with ID starting with 7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8 not found: ID does not exist"
Jul 15 04:09:08.441911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.441907    2524 scope.go:117] "RemoveContainer" containerID="9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"
Jul 15 04:09:08.442127 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.442107    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d"} err="failed to get container status \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": rpc error: code = NotFound desc = could not find container \"9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d\": container with ID starting with 9febb65d5f6cc2e77ff89f82a108fe4e5a328916de1c63cefe9847ef3f4f806d not found: ID does not exist"
Jul 15 04:09:08.442127 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.442127    2524 scope.go:117] "RemoveContainer" containerID="19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"
Jul 15 04:09:08.442408 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.442381    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483"} err="failed to get container status \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": rpc error: code = NotFound desc = could not find container \"19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483\": container with ID starting with 19d140fd43851a42b87d9ccece516bdd8e874fee9989b6402c9a1a00eb591483 not found: ID does not exist"
Jul 15 04:09:08.442517 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.442410    2524 scope.go:117] "RemoveContainer" containerID="7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"
Jul 15 04:09:08.442695 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.442675    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006"} err="failed to get container status \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": rpc error: code = NotFound desc = could not find container \"7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006\": container with ID starting with 7984ec74a97a32d74029c021a3373665ca9adb47fc420f0891a988ac930b5006 not found: ID does not exist"
Jul 15 04:09:08.442768 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.442697    2524 scope.go:117] "RemoveContainer" containerID="faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"
Jul 15 04:09:08.443300 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.443281    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 04:09:08.443379 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.443331    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af"} err="failed to get container status \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": rpc error: code = NotFound desc = could not find container \"faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af\": container with ID starting with faf1278206b229dac41038be511e0d4431b80afbd654fbc0cbf800e06b2f22af not found: ID does not exist"
Jul 15 04:09:08.443379 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.443362    2524 scope.go:117] "RemoveContainer" containerID="c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"
Jul 15 04:09:08.443684 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.443663    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1"} err="failed to get container status \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": rpc error: code = NotFound desc = could not find container \"c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1\": container with ID starting with c126f00ad0f30b850f80180fc56f790ed3da3b203fc1461d1ee99269389a8fe1 not found: ID does not exist"
Jul 15 04:09:08.443684 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.443684    2524 scope.go:117] "RemoveContainer" containerID="2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"
Jul 15 04:09:08.443959 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.443932    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc"} err="failed to get container status \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": rpc error: code = NotFound desc = could not find container \"2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc\": container with ID starting with 2290951ad7d1d25b86bf5c2e11d9a1cd95f958c5078a5226f9f803ecc5b81ccc not found: ID does not exist"
Jul 15 04:09:08.444045 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.443959    2524 scope.go:117] "RemoveContainer" containerID="7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"
Jul 15 04:09:08.444226 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.444203    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8"} err="failed to get container status \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": rpc error: code = NotFound desc = could not find container \"7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8\": container with ID starting with 7f5d20a83508ef372de7264f1cb76208332d721dfce1f3162825b6aacdc2a0c8 not found: ID does not exist"
Jul 15 04:09:08.445176 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.445161    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 04:09:08.563005 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.562967    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563005 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563007    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563027    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563050    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563068    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563085    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563102    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563115    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qrhb\" (UniqueName: \"kubernetes.io/projected/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-kube-api-access-2qrhb\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563140    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563155    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563169    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563183    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-web-config\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563206    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-config-out\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563586 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563242    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563586 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563276    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563586 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563295    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563586 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563329    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563586 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563365    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-config\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563586 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563399    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.563586 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.563426    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.663802 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.663761    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.663922 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.663805    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.663990 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.663960    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664048 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664018    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664102 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664050    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664102 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664080    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664201 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664106    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qrhb\" (UniqueName: \"kubernetes.io/projected/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-kube-api-access-2qrhb\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664736 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664700    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664736 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664722    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664782    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664815    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664822    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664844    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664864    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-web-config\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.664911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664911    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-config-out\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.665229 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664939    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.665229 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664969    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.665229 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.664997    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.665229 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.665021    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.665229 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.665052    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-config\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.665229 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.665079    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.665229 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.665112    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.665229 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.665161    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.665229 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.665174    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.665667 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.665336    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.665667 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.665418    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.666139 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.665843    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.667254 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.667229    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.667392 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.667274    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.668402 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.668377    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.668711 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.668682    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.668802 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.668736    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-config\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.668802 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.668796    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-config-out\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.669623 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.669571    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-web-config\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.669888 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.669815    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.670232 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.670209    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.670343 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.670272    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.671181 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.671150    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.671181 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.671169    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.672374 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.672353    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qrhb\" (UniqueName: \"kubernetes.io/projected/16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8-kube-api-access-2qrhb\") pod \"prometheus-k8s-0\" (UID: \"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.742915 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.742874    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:08.885547 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:08.885520    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 04:09:08.887403 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:09:08.887381    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ef1fc5_bfa6_473a_8ccf_e5449ac1d7e8.slice/crio-53a41bdc4351374676013432436d19a153f147a65ce2ee5a4f9b9d32b02ab056 WatchSource:0}: Error finding container 53a41bdc4351374676013432436d19a153f147a65ce2ee5a4f9b9d32b02ab056: Status 404 returned error can't find the container with id 53a41bdc4351374676013432436d19a153f147a65ce2ee5a4f9b9d32b02ab056
Jul 15 04:09:09.380542 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:09.380515    2524 generic.go:356] "Generic (PLEG): container finished" podID="16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8" containerID="40240cdae9d570711be944b943e9ad0b109c560663b87ae582bee8da8595b21e" exitCode=0
Jul 15 04:09:09.381015 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:09.380572    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8","Type":"ContainerDied","Data":"40240cdae9d570711be944b943e9ad0b109c560663b87ae582bee8da8595b21e"}
Jul 15 04:09:09.381015 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:09.380657    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8","Type":"ContainerStarted","Data":"53a41bdc4351374676013432436d19a153f147a65ce2ee5a4f9b9d32b02ab056"}
Jul 15 04:09:09.431690 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:09.431664    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="768ff916-ac83-4ead-9382-cadd8c57e0ba" path="/var/lib/kubelet/pods/768ff916-ac83-4ead-9382-cadd8c57e0ba/volumes"
Jul 15 04:09:10.389261 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.389225    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8","Type":"ContainerStarted","Data":"23af6961adbe26ad201ff5616059e6b62163bb5fdee45b7e685be0b6487909b8"}
Jul 15 04:09:10.389786 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.389343    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8","Type":"ContainerStarted","Data":"96ce07c093801815686b02d03b6453d023b400283650d89bf1ab6a256c5345ae"}
Jul 15 04:09:10.389786 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.389390    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8","Type":"ContainerStarted","Data":"19963e6fca677a78f4ad6529843c8f4c3c0c1c96ad6231eab90fddd90376a801"}
Jul 15 04:09:10.389786 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.389433    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8","Type":"ContainerStarted","Data":"fe310dbae96422a48f21ddf7308d215c6c2254fc8168cf6076a720f895e62867"}
Jul 15 04:09:10.389786 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.389469    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8","Type":"ContainerStarted","Data":"26e18f703150237f0d3a1b3ee8ebb523f18c8aaea32df52bf88fe792655f7d6b"}
Jul 15 04:09:10.389786 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.389507    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8","Type":"ContainerStarted","Data":"b99e920d9050d40317e3a1989ce98400a4d8308580043460e6574b7aeb5219dc"}
Jul 15 04:09:10.416521 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.416463    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.393418575 podStartE2EDuration="2.416447552s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.028977ms" isStatefulPod=false podCreationTimestamp="2026-07-15 04:09:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:09:10.415855994 +0000 UTC m=+179.727237243" watchObservedRunningTime="2026-07-15 04:09:10.416447552 +0000 UTC m=+179.727828800"
Jul 15 04:09:10.715443 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.715367    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5954dffc46-s9hlr"]
Jul 15 04:09:10.718267 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.718249    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.729649 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.729624    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5954dffc46-s9hlr"]
Jul 15 04:09:10.784900 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.784858    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-oauth-config\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.785119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.784916    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-oauth-serving-cert\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.785119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.784976    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-console-config\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.785119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.785002    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpqkb\" (UniqueName: \"kubernetes.io/projected/f329bf3d-a794-49fd-99c4-174541b1a58b-kube-api-access-mpqkb\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.785119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.785038    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-service-ca\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.785119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.785096    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-serving-cert\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.785313 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.785126    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-trusted-ca-bundle\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.886184 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.886141    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-oauth-config\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.886356 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.886204    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-oauth-serving-cert\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.886356 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.886238    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-console-config\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.886356 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.886259    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpqkb\" (UniqueName: \"kubernetes.io/projected/f329bf3d-a794-49fd-99c4-174541b1a58b-kube-api-access-mpqkb\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.886356 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.886294    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-service-ca\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.886356 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.886330    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-serving-cert\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.886640 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.886357    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-trusted-ca-bundle\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.887133 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.887099    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-console-config\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.887133 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.887123    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-service-ca\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.887290 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.887151    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-oauth-serving-cert\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.887347 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.887309    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-trusted-ca-bundle\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.889294 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.889272    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-oauth-config\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.889489 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.889467    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-serving-cert\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:10.893832 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:10.893812    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpqkb\" (UniqueName: \"kubernetes.io/projected/f329bf3d-a794-49fd-99c4-174541b1a58b-kube-api-access-mpqkb\") pod \"console-5954dffc46-s9hlr\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") " pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:11.027842 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:11.027754    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:11.153819 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:11.153793    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5954dffc46-s9hlr"]
Jul 15 04:09:11.156016 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:09:11.155980    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf329bf3d_a794_49fd_99c4_174541b1a58b.slice/crio-60c5371aa7847ece481140f2b317dd4b873a42b57db3fadd60b4be91803213bf WatchSource:0}: Error finding container 60c5371aa7847ece481140f2b317dd4b873a42b57db3fadd60b4be91803213bf: Status 404 returned error can't find the container with id 60c5371aa7847ece481140f2b317dd4b873a42b57db3fadd60b4be91803213bf
Jul 15 04:09:11.394681 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:11.394648    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5954dffc46-s9hlr" event={"ID":"f329bf3d-a794-49fd-99c4-174541b1a58b","Type":"ContainerStarted","Data":"3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611"}
Jul 15 04:09:11.395077 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:11.394699    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5954dffc46-s9hlr" event={"ID":"f329bf3d-a794-49fd-99c4-174541b1a58b","Type":"ContainerStarted","Data":"60c5371aa7847ece481140f2b317dd4b873a42b57db3fadd60b4be91803213bf"}
Jul 15 04:09:11.414066 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:11.414013    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5954dffc46-s9hlr" podStartSLOduration=1.4139972250000001 podStartE2EDuration="1.413997225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:09:11.412692353 +0000 UTC m=+180.724073601" watchObservedRunningTime="2026-07-15 04:09:11.413997225 +0000 UTC m=+180.725378873"
Jul 15 04:09:13.743227 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:13.743198    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:09:21.028270 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:21.028229    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:21.028917 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:21.028332    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:21.033215 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:21.033192    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:21.435933 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:21.435908    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:09:21.480811 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:21.480780    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-855b66cb6-kw6ft"]
Jul 15 04:09:46.508204 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.508134    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-855b66cb6-kw6ft" podUID="6560857a-3d00-4b36-a8da-c6c32a8d7ff7" containerName="console" containerID="cri-o://ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5" gracePeriod=15
Jul 15 04:09:46.749511 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.749488    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:09:46.789006 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.788936    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-oauth-config\") pod \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") "
Jul 15 04:09:46.789006 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.788982    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-config\") pod \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") "
Jul 15 04:09:46.789006 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.789001    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-serving-cert\") pod \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") "
Jul 15 04:09:46.789210 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.789183    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-service-ca\") pod \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") "
Jul 15 04:09:46.789257 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.789225    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-trusted-ca-bundle\") pod \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") "
Jul 15 04:09:46.789303 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.789287    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-oauth-serving-cert\") pod \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") "
Jul 15 04:09:46.789361 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.789344    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4gwj\" (UniqueName: \"kubernetes.io/projected/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-kube-api-access-p4gwj\") pod \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\" (UID: \"6560857a-3d00-4b36-a8da-c6c32a8d7ff7\") "
Jul 15 04:09:46.789475 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.789451    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-config" (OuterVolumeSpecName: "console-config") pod "6560857a-3d00-4b36-a8da-c6c32a8d7ff7" (UID: "6560857a-3d00-4b36-a8da-c6c32a8d7ff7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:09:46.789846 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.789629    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-service-ca" (OuterVolumeSpecName: "service-ca") pod "6560857a-3d00-4b36-a8da-c6c32a8d7ff7" (UID: "6560857a-3d00-4b36-a8da-c6c32a8d7ff7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:09:46.789846 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.789653    2524 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:46.789846 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.789669    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6560857a-3d00-4b36-a8da-c6c32a8d7ff7" (UID: "6560857a-3d00-4b36-a8da-c6c32a8d7ff7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:09:46.789846 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.789674    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6560857a-3d00-4b36-a8da-c6c32a8d7ff7" (UID: "6560857a-3d00-4b36-a8da-c6c32a8d7ff7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:09:46.791270 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.791238    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6560857a-3d00-4b36-a8da-c6c32a8d7ff7" (UID: "6560857a-3d00-4b36-a8da-c6c32a8d7ff7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:09:46.791270 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.791265    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6560857a-3d00-4b36-a8da-c6c32a8d7ff7" (UID: "6560857a-3d00-4b36-a8da-c6c32a8d7ff7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:09:46.791451 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.791428    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-kube-api-access-p4gwj" (OuterVolumeSpecName: "kube-api-access-p4gwj") pod "6560857a-3d00-4b36-a8da-c6c32a8d7ff7" (UID: "6560857a-3d00-4b36-a8da-c6c32a8d7ff7"). InnerVolumeSpecName "kube-api-access-p4gwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:09:46.891046 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.891000    2524 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:46.891046 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.891039    2524 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-trusted-ca-bundle\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:46.891046 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.891049    2524 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-oauth-serving-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:46.891046 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.891060    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4gwj\" (UniqueName: \"kubernetes.io/projected/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-kube-api-access-p4gwj\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:46.891046 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.891068    2524 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-oauth-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:46.891337 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:46.891077    2524 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6560857a-3d00-4b36-a8da-c6c32a8d7ff7-console-serving-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:09:47.525986 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:47.525944    2524 generic.go:356] "Generic (PLEG): container finished" podID="6560857a-3d00-4b36-a8da-c6c32a8d7ff7" containerID="ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5" exitCode=0
Jul 15 04:09:47.526391 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:47.525994    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855b66cb6-kw6ft"
Jul 15 04:09:47.526391 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:47.526038    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855b66cb6-kw6ft" event={"ID":"6560857a-3d00-4b36-a8da-c6c32a8d7ff7","Type":"ContainerDied","Data":"ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5"}
Jul 15 04:09:47.526391 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:47.526090    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855b66cb6-kw6ft" event={"ID":"6560857a-3d00-4b36-a8da-c6c32a8d7ff7","Type":"ContainerDied","Data":"063fec29a51a49fd0009cb2894e528ca8a5c4400bdb543f11b9f5c7948dffb20"}
Jul 15 04:09:47.526391 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:47.526114    2524 scope.go:117] "RemoveContainer" containerID="ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5"
Jul 15 04:09:47.534205 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:47.534186    2524 scope.go:117] "RemoveContainer" containerID="ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5"
Jul 15 04:09:47.534457 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:09:47.534438    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5\": container with ID starting with ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5 not found: ID does not exist" containerID="ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5"
Jul 15 04:09:47.534508 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:47.534465    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5"} err="failed to get container status \"ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5\": rpc error: code = NotFound desc = could not find container \"ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5\": container with ID starting with ec6064b166beb9ff3b6f03f8022d5ccaf99670c119a5b3abe41da50bc90433f5 not found: ID does not exist"
Jul 15 04:09:47.542823 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:47.542800    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-855b66cb6-kw6ft"]
Jul 15 04:09:47.548006 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:47.547986    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-855b66cb6-kw6ft"]
Jul 15 04:09:49.431030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:09:49.430999    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6560857a-3d00-4b36-a8da-c6c32a8d7ff7" path="/var/lib/kubelet/pods/6560857a-3d00-4b36-a8da-c6c32a8d7ff7/volumes"
Jul 15 04:10:08.743998 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:08.743950    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:10:08.760115 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:08.760088    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:10:09.627820 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:09.627794    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 04:10:17.534335 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.534297    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57c44979fd-98d6d"]
Jul 15 04:10:17.534792 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.534569    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6560857a-3d00-4b36-a8da-c6c32a8d7ff7" containerName="console"
Jul 15 04:10:17.534792 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.534580    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="6560857a-3d00-4b36-a8da-c6c32a8d7ff7" containerName="console"
Jul 15 04:10:17.534792 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.534670    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="6560857a-3d00-4b36-a8da-c6c32a8d7ff7" containerName="console"
Jul 15 04:10:17.537319 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.537296    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.547699 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.547678    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57c44979fd-98d6d"]
Jul 15 04:10:17.648445 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.648406    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-serving-cert\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.648653 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.648459    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-service-ca\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.648653 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.648478    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-oauth-config\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.648653 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.648493    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-oauth-serving-cert\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.648653 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.648512    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kfvx\" (UniqueName: \"kubernetes.io/projected/4f65fd5f-3605-4023-ab0e-4d3badf2329c-kube-api-access-2kfvx\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.648653 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.648529    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-config\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.648653 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.648547    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-trusted-ca-bundle\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.749541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.749503    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kfvx\" (UniqueName: \"kubernetes.io/projected/4f65fd5f-3605-4023-ab0e-4d3badf2329c-kube-api-access-2kfvx\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.749541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.749540    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-config\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.749763 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.749557    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-trusted-ca-bundle\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.749763 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.749653    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-serving-cert\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.749763 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.749709    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-service-ca\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.749763 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.749732    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-oauth-config\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.749763 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.749753    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-oauth-serving-cert\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.750349 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.750323    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-config\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.750472 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.750407    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-oauth-serving-cert\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.750472 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.750411    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-service-ca\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.750472 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.750456    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-trusted-ca-bundle\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.752327 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.752302    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-oauth-config\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.752421 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.752383    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-serving-cert\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.756882 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.756850    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kfvx\" (UniqueName: \"kubernetes.io/projected/4f65fd5f-3605-4023-ab0e-4d3badf2329c-kube-api-access-2kfvx\") pod \"console-57c44979fd-98d6d\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") " pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.847007 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.846928    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:17.971728 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:17.971705    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57c44979fd-98d6d"]
Jul 15 04:10:17.974335 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:10:17.974307    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f65fd5f_3605_4023_ab0e_4d3badf2329c.slice/crio-0551f3389852f873d469923b86d8e7a3ff06db573c88d03e31f211390ff5f6a9 WatchSource:0}: Error finding container 0551f3389852f873d469923b86d8e7a3ff06db573c88d03e31f211390ff5f6a9: Status 404 returned error can't find the container with id 0551f3389852f873d469923b86d8e7a3ff06db573c88d03e31f211390ff5f6a9
Jul 15 04:10:18.646878 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:18.646842    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c44979fd-98d6d" event={"ID":"4f65fd5f-3605-4023-ab0e-4d3badf2329c","Type":"ContainerStarted","Data":"3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c"}
Jul 15 04:10:18.647250 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:18.646896    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c44979fd-98d6d" event={"ID":"4f65fd5f-3605-4023-ab0e-4d3badf2329c","Type":"ContainerStarted","Data":"0551f3389852f873d469923b86d8e7a3ff06db573c88d03e31f211390ff5f6a9"}
Jul 15 04:10:18.663502 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:18.663453    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57c44979fd-98d6d" podStartSLOduration=1.663438813 podStartE2EDuration="1.663438813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:10:18.663175485 +0000 UTC m=+247.974556741" watchObservedRunningTime="2026-07-15 04:10:18.663438813 +0000 UTC m=+247.974820060"
Jul 15 04:10:27.847430 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:27.847397    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:27.847979 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:27.847448    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:27.851923 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:27.851898    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:28.686197 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:28.686169    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:10:28.732140 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:28.732112    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5954dffc46-s9hlr"]
Jul 15 04:10:53.759328 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:53.759267    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5954dffc46-s9hlr" podUID="f329bf3d-a794-49fd-99c4-174541b1a58b" containerName="console" containerID="cri-o://3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611" gracePeriod=15
Jul 15 04:10:53.999517 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:53.999488    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:10:54.147170 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.147145    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-oauth-serving-cert\") pod \"f329bf3d-a794-49fd-99c4-174541b1a58b\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") "
Jul 15 04:10:54.147307 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.147176    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpqkb\" (UniqueName: \"kubernetes.io/projected/f329bf3d-a794-49fd-99c4-174541b1a58b-kube-api-access-mpqkb\") pod \"f329bf3d-a794-49fd-99c4-174541b1a58b\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") "
Jul 15 04:10:54.147307 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.147201    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-trusted-ca-bundle\") pod \"f329bf3d-a794-49fd-99c4-174541b1a58b\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") "
Jul 15 04:10:54.147307 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.147236    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-serving-cert\") pod \"f329bf3d-a794-49fd-99c4-174541b1a58b\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") "
Jul 15 04:10:54.147307 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.147269    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-oauth-config\") pod \"f329bf3d-a794-49fd-99c4-174541b1a58b\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") "
Jul 15 04:10:54.147307 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.147284    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-console-config\") pod \"f329bf3d-a794-49fd-99c4-174541b1a58b\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") "
Jul 15 04:10:54.147562 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.147317    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-service-ca\") pod \"f329bf3d-a794-49fd-99c4-174541b1a58b\" (UID: \"f329bf3d-a794-49fd-99c4-174541b1a58b\") "
Jul 15 04:10:54.147746 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.147640    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f329bf3d-a794-49fd-99c4-174541b1a58b" (UID: "f329bf3d-a794-49fd-99c4-174541b1a58b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:10:54.147746 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.147734    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f329bf3d-a794-49fd-99c4-174541b1a58b" (UID: "f329bf3d-a794-49fd-99c4-174541b1a58b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:10:54.147856 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.147746    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-console-config" (OuterVolumeSpecName: "console-config") pod "f329bf3d-a794-49fd-99c4-174541b1a58b" (UID: "f329bf3d-a794-49fd-99c4-174541b1a58b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:10:54.147856 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.147845    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-service-ca" (OuterVolumeSpecName: "service-ca") pod "f329bf3d-a794-49fd-99c4-174541b1a58b" (UID: "f329bf3d-a794-49fd-99c4-174541b1a58b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:10:54.149345 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.149323    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f329bf3d-a794-49fd-99c4-174541b1a58b-kube-api-access-mpqkb" (OuterVolumeSpecName: "kube-api-access-mpqkb") pod "f329bf3d-a794-49fd-99c4-174541b1a58b" (UID: "f329bf3d-a794-49fd-99c4-174541b1a58b"). InnerVolumeSpecName "kube-api-access-mpqkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:10:54.149908 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.149887    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f329bf3d-a794-49fd-99c4-174541b1a58b" (UID: "f329bf3d-a794-49fd-99c4-174541b1a58b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:10:54.149995 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.149902    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f329bf3d-a794-49fd-99c4-174541b1a58b" (UID: "f329bf3d-a794-49fd-99c4-174541b1a58b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:10:54.248269 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.248228    2524 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-oauth-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:10:54.248269 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.248263    2524 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-console-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:10:54.248269 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.248276    2524 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:10:54.248487 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.248287    2524 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-oauth-serving-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:10:54.248487 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.248300    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpqkb\" (UniqueName: \"kubernetes.io/projected/f329bf3d-a794-49fd-99c4-174541b1a58b-kube-api-access-mpqkb\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:10:54.248487 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.248312    2524 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f329bf3d-a794-49fd-99c4-174541b1a58b-trusted-ca-bundle\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:10:54.248487 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.248323    2524 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f329bf3d-a794-49fd-99c4-174541b1a58b-console-serving-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:10:54.776841 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.776803    2524 generic.go:356] "Generic (PLEG): container finished" podID="f329bf3d-a794-49fd-99c4-174541b1a58b" containerID="3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611" exitCode=0
Jul 15 04:10:54.777289 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.776848    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5954dffc46-s9hlr"
Jul 15 04:10:54.777289 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.776856    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5954dffc46-s9hlr" event={"ID":"f329bf3d-a794-49fd-99c4-174541b1a58b","Type":"ContainerDied","Data":"3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611"}
Jul 15 04:10:54.777289 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.776906    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5954dffc46-s9hlr" event={"ID":"f329bf3d-a794-49fd-99c4-174541b1a58b","Type":"ContainerDied","Data":"60c5371aa7847ece481140f2b317dd4b873a42b57db3fadd60b4be91803213bf"}
Jul 15 04:10:54.777289 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.776937    2524 scope.go:117] "RemoveContainer" containerID="3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611"
Jul 15 04:10:54.785138 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.785122    2524 scope.go:117] "RemoveContainer" containerID="3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611"
Jul 15 04:10:54.785382 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:10:54.785365    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611\": container with ID starting with 3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611 not found: ID does not exist" containerID="3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611"
Jul 15 04:10:54.785441 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.785394    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611"} err="failed to get container status \"3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611\": rpc error: code = NotFound desc = could not find container \"3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611\": container with ID starting with 3b25b2da1c0449f512240d7f6579ffd9ed6fb9730346a7bb58a01d8c13756611 not found: ID does not exist"
Jul 15 04:10:54.796023 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.796000    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5954dffc46-s9hlr"]
Jul 15 04:10:54.799640 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:54.799620    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5954dffc46-s9hlr"]
Jul 15 04:10:55.430744 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:10:55.430705    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f329bf3d-a794-49fd-99c4-174541b1a58b" path="/var/lib/kubelet/pods/f329bf3d-a794-49fd-99c4-174541b1a58b/volumes"
Jul 15 04:11:11.270307 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:11:11.270277    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:11:11.275371 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:11:11.275351    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:11:11.276111 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:11:11.276091    2524 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 04:12:20.679457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.679383    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx"]
Jul 15 04:12:20.679896 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.679681    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f329bf3d-a794-49fd-99c4-174541b1a58b" containerName="console"
Jul 15 04:12:20.679896 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.679692    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="f329bf3d-a794-49fd-99c4-174541b1a58b" containerName="console"
Jul 15 04:12:20.679896 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.679752    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="f329bf3d-a794-49fd-99c4-174541b1a58b" containerName="console"
Jul 15 04:12:20.682460 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.682444    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx"
Jul 15 04:12:20.684754 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.684732    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 04:12:20.684925 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.684907    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 04:12:20.685687 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.685671    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-8dgj8\""
Jul 15 04:12:20.691198 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.691174    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx"]
Jul 15 04:12:20.793497 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.793456    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6215b415-368a-4839-ba13-33014c84a73e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4dsvx\" (UID: \"6215b415-368a-4839-ba13-33014c84a73e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx"
Jul 15 04:12:20.793697 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.793509    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k875g\" (UniqueName: \"kubernetes.io/projected/6215b415-368a-4839-ba13-33014c84a73e-kube-api-access-k875g\") pod \"openshift-lws-operator-fd8ccff4c-4dsvx\" (UID: \"6215b415-368a-4839-ba13-33014c84a73e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx"
Jul 15 04:12:20.894378 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.894338    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6215b415-368a-4839-ba13-33014c84a73e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4dsvx\" (UID: \"6215b415-368a-4839-ba13-33014c84a73e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx"
Jul 15 04:12:20.894577 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.894390    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k875g\" (UniqueName: \"kubernetes.io/projected/6215b415-368a-4839-ba13-33014c84a73e-kube-api-access-k875g\") pod \"openshift-lws-operator-fd8ccff4c-4dsvx\" (UID: \"6215b415-368a-4839-ba13-33014c84a73e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx"
Jul 15 04:12:20.894753 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.894732    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6215b415-368a-4839-ba13-33014c84a73e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4dsvx\" (UID: \"6215b415-368a-4839-ba13-33014c84a73e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx"
Jul 15 04:12:20.902059 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.902024    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k875g\" (UniqueName: \"kubernetes.io/projected/6215b415-368a-4839-ba13-33014c84a73e-kube-api-access-k875g\") pod \"openshift-lws-operator-fd8ccff4c-4dsvx\" (UID: \"6215b415-368a-4839-ba13-33014c84a73e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx"
Jul 15 04:12:20.991822 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:20.991732    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx"
Jul 15 04:12:21.122175 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:21.122106    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx"]
Jul 15 04:12:21.124400 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:12:21.124376    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6215b415_368a_4839_ba13_33014c84a73e.slice/crio-b9475549ed10b65164fae4d0ad8feb202f9a0fc6876617367a9f064264776254 WatchSource:0}: Error finding container b9475549ed10b65164fae4d0ad8feb202f9a0fc6876617367a9f064264776254: Status 404 returned error can't find the container with id b9475549ed10b65164fae4d0ad8feb202f9a0fc6876617367a9f064264776254
Jul 15 04:12:21.125781 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:21.125761    2524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 04:12:22.099947 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:22.099900    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx" event={"ID":"6215b415-368a-4839-ba13-33014c84a73e","Type":"ContainerStarted","Data":"b9475549ed10b65164fae4d0ad8feb202f9a0fc6876617367a9f064264776254"}
Jul 15 04:12:25.114368 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:25.114333    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx" event={"ID":"6215b415-368a-4839-ba13-33014c84a73e","Type":"ContainerStarted","Data":"66dc67e88138e3d70ff81a1b91fdf6bd1d4f4e9c00d91b58db9aecbda9e5a66d"}
Jul 15 04:12:25.129512 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:25.129462    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4dsvx" podStartSLOduration=1.308721943 podStartE2EDuration="5.129446173s" totalImagesPullingTime="3.82072423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:12:25.127960139 +0000 UTC m=+374.439341386" watchObservedRunningTime="2026-07-15 04:12:25.129446173 +0000 UTC m=+374.440827422"
Jul 15 04:12:44.868908 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.868870    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"]
Jul 15 04:12:44.872272 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.872250    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:44.876055 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.875996    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 04:12:44.876184 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.876087    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 04:12:44.876331 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.876309    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5c9wl\""
Jul 15 04:12:44.876470 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.876400    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 04:12:44.876527 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.876469    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 04:12:44.897932 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.897905    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:44.898064 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.897943    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-tmp\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:44.898064 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.897961    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:44.898144 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.898056    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-manifests\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:44.898144 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.898094    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkmpd\" (UniqueName: \"kubernetes.io/projected/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-kube-api-access-xkmpd\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:44.911848 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.911820    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"]
Jul 15 04:12:44.998489 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.998447    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:44.998688 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.998496    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-tmp\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:44.998688 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.998527    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:44.998688 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.998586    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-manifests\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:44.998688 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.998627    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkmpd\" (UniqueName: \"kubernetes.io/projected/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-kube-api-access-xkmpd\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:44.999046 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:44.999019    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-manifests\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:45.000945 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:45.000927    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-tmp\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:45.001233 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:45.001214    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:45.001367 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:45.001345    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:45.008579 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:45.008550    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkmpd\" (UniqueName: \"kubernetes.io/projected/863afdaf-88d5-467f-aaa4-c0a1af6bbfe8-kube-api-access-xkmpd\") pod \"opendatahub-operator-controller-manager-6dddd85b7f-dl9pg\" (UID: \"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:45.182419 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:45.182327    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:45.315349 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:45.315327    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"]
Jul 15 04:12:45.318126 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:12:45.318099    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod863afdaf_88d5_467f_aaa4_c0a1af6bbfe8.slice/crio-23e0b20ee80b6ed222bf2af54d8e8ebb00f085e1d188e46d1f3ae090d324c6c9 WatchSource:0}: Error finding container 23e0b20ee80b6ed222bf2af54d8e8ebb00f085e1d188e46d1f3ae090d324c6c9: Status 404 returned error can't find the container with id 23e0b20ee80b6ed222bf2af54d8e8ebb00f085e1d188e46d1f3ae090d324c6c9
Jul 15 04:12:46.202049 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:46.202011    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg" event={"ID":"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8","Type":"ContainerStarted","Data":"23e0b20ee80b6ed222bf2af54d8e8ebb00f085e1d188e46d1f3ae090d324c6c9"}
Jul 15 04:12:49.218743 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:49.218707    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg" event={"ID":"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8","Type":"ContainerStarted","Data":"c274741977f821d82f742d53e45a513b70904a22932ac6e59307a8587a4adfcb"}
Jul 15 04:12:50.224573 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:50.224537    2524 generic.go:356] "Generic (PLEG): container finished" podID="863afdaf-88d5-467f-aaa4-c0a1af6bbfe8" containerID="c274741977f821d82f742d53e45a513b70904a22932ac6e59307a8587a4adfcb" exitCode=0
Jul 15 04:12:50.224989 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:50.224626    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg" event={"ID":"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8","Type":"ContainerDied","Data":"c274741977f821d82f742d53e45a513b70904a22932ac6e59307a8587a4adfcb"}
Jul 15 04:12:51.230520 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:51.230484    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg" event={"ID":"863afdaf-88d5-467f-aaa4-c0a1af6bbfe8","Type":"ContainerStarted","Data":"43330d4bc849e3e47f44cd6e55e8f98c8bf7eb73f9a8d9512011e3fb36c068e1"}
Jul 15 04:12:51.231010 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:51.230612    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:12:51.252389 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:12:51.252341    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg" podStartSLOduration=2.83052007 podStartE2EDuration="7.252325463s" totalImagesPullingTime="3.410155852s" totalInitContainerRuntime="1.011649541s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:12:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:12:51.251295231 +0000 UTC m=+400.562676490" watchObservedRunningTime="2026-07-15 04:12:51.252325463 +0000 UTC m=+400.563706709"
Jul 15 04:13:02.237388 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:02.237355    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dddd85b7f-dl9pg"
Jul 15 04:13:10.368763 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.368728    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"]
Jul 15 04:13:10.372136 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.372114    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.379660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.378195    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 15 04:13:10.379660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.378432    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 15 04:13:10.379660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.378668    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-w6j9x\""
Jul 15 04:13:10.379660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.378894    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 15 04:13:10.383791 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.383758    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"]
Jul 15 04:13:10.516156 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.516122    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d6lk\" (UniqueName: \"kubernetes.io/projected/2823f9a0-14b6-41b6-aa5d-476f603c1267-kube-api-access-8d6lk\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.516329 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.516172    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2823f9a0-14b6-41b6-aa5d-476f603c1267-cert\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.516329 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.516201    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2823f9a0-14b6-41b6-aa5d-476f603c1267-metrics-cert\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.516329 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.516253    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2823f9a0-14b6-41b6-aa5d-476f603c1267-manager-config\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.617232 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.617198    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2823f9a0-14b6-41b6-aa5d-476f603c1267-manager-config\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.617407 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.617249    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d6lk\" (UniqueName: \"kubernetes.io/projected/2823f9a0-14b6-41b6-aa5d-476f603c1267-kube-api-access-8d6lk\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.617407 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.617299    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2823f9a0-14b6-41b6-aa5d-476f603c1267-cert\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.617407 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.617333    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2823f9a0-14b6-41b6-aa5d-476f603c1267-metrics-cert\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.617997 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.617974    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2823f9a0-14b6-41b6-aa5d-476f603c1267-manager-config\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.620013 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.619992    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2823f9a0-14b6-41b6-aa5d-476f603c1267-cert\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.620099 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.620015    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2823f9a0-14b6-41b6-aa5d-476f603c1267-metrics-cert\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.630956 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.630930    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d6lk\" (UniqueName: \"kubernetes.io/projected/2823f9a0-14b6-41b6-aa5d-476f603c1267-kube-api-access-8d6lk\") pod \"lws-controller-manager-c845c8db6-f69db\" (UID: \"2823f9a0-14b6-41b6-aa5d-476f603c1267\") " pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.685682 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.685645    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:10.819584 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:10.819560    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"]
Jul 15 04:13:10.821986 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:13:10.821960    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2823f9a0_14b6_41b6_aa5d_476f603c1267.slice/crio-b8491abc06a4c4151e791bf0a31537bcb203ff80181d1ca28a3c76d57cbbcf4a WatchSource:0}: Error finding container b8491abc06a4c4151e791bf0a31537bcb203ff80181d1ca28a3c76d57cbbcf4a: Status 404 returned error can't find the container with id b8491abc06a4c4151e791bf0a31537bcb203ff80181d1ca28a3c76d57cbbcf4a
Jul 15 04:13:11.331340 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:11.331306    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db" event={"ID":"2823f9a0-14b6-41b6-aa5d-476f603c1267","Type":"ContainerStarted","Data":"b8491abc06a4c4151e791bf0a31537bcb203ff80181d1ca28a3c76d57cbbcf4a"}
Jul 15 04:13:12.460685 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.460649    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"]
Jul 15 04:13:12.464014 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.463997    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.466524 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.466491    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 15 04:13:12.466796 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.466774    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nbcpr\""
Jul 15 04:13:12.466887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.466781    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 04:13:12.466887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.466824    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 15 04:13:12.466986 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.466915    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 15 04:13:12.467112 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.467097    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 04:13:12.467418 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.467403    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 04:13:12.478034 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.478013    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"]
Jul 15 04:13:12.533955 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.533920    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b825790a-c79c-472d-9c56-ac3a248491bc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.534145 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.533965    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b825790a-c79c-472d-9c56-ac3a248491bc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.534145 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.533996    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b825790a-c79c-472d-9c56-ac3a248491bc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.534145 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.534072    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhdgv\" (UniqueName: \"kubernetes.io/projected/b825790a-c79c-472d-9c56-ac3a248491bc-kube-api-access-dhdgv\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.534145 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.534105    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b825790a-c79c-472d-9c56-ac3a248491bc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.534145 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.534126    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b825790a-c79c-472d-9c56-ac3a248491bc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.534358 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.534211    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b825790a-c79c-472d-9c56-ac3a248491bc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.634980 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.634938    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhdgv\" (UniqueName: \"kubernetes.io/projected/b825790a-c79c-472d-9c56-ac3a248491bc-kube-api-access-dhdgv\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.634980 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.634988    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b825790a-c79c-472d-9c56-ac3a248491bc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.635225 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.635007    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b825790a-c79c-472d-9c56-ac3a248491bc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.635225 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.635054    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b825790a-c79c-472d-9c56-ac3a248491bc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.635225 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.635092    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b825790a-c79c-472d-9c56-ac3a248491bc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.635365 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.635221    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b825790a-c79c-472d-9c56-ac3a248491bc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.635365 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.635260    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b825790a-c79c-472d-9c56-ac3a248491bc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.635864 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.635789    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b825790a-c79c-472d-9c56-ac3a248491bc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.637919 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.637890    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b825790a-c79c-472d-9c56-ac3a248491bc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.638040 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.637898    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b825790a-c79c-472d-9c56-ac3a248491bc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.638040 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.637966    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b825790a-c79c-472d-9c56-ac3a248491bc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.638856 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.638830    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b825790a-c79c-472d-9c56-ac3a248491bc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.646244 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.646207    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b825790a-c79c-472d-9c56-ac3a248491bc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.646488 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.646465    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhdgv\" (UniqueName: \"kubernetes.io/projected/b825790a-c79c-472d-9c56-ac3a248491bc-kube-api-access-dhdgv\") pod \"istiod-openshift-gateway-94bb8fbfd-95pfs\" (UID: \"b825790a-c79c-472d-9c56-ac3a248491bc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.773429 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.773345    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:12.926556 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:12.926528    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"]
Jul 15 04:13:12.929091 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:13:12.929061    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb825790a_c79c_472d_9c56_ac3a248491bc.slice/crio-9c267871735428cf59548bc0a2723995714d311bec2a5e3aae7e09594265c0f6 WatchSource:0}: Error finding container 9c267871735428cf59548bc0a2723995714d311bec2a5e3aae7e09594265c0f6: Status 404 returned error can't find the container with id 9c267871735428cf59548bc0a2723995714d311bec2a5e3aae7e09594265c0f6
Jul 15 04:13:13.343016 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:13.342976    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs" event={"ID":"b825790a-c79c-472d-9c56-ac3a248491bc","Type":"ContainerStarted","Data":"9c267871735428cf59548bc0a2723995714d311bec2a5e3aae7e09594265c0f6"}
Jul 15 04:13:14.350258 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:14.350214    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db" event={"ID":"2823f9a0-14b6-41b6-aa5d-476f603c1267","Type":"ContainerStarted","Data":"582bd23f03966fdc4ff73ea25adaaf97be3a4b42b3be5a1a20e82c94d840c1a1"}
Jul 15 04:13:14.350790 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:14.350393    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:13:14.412026 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:14.411966    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db" podStartSLOduration=1.675225802 podStartE2EDuration="4.411942725s" totalImagesPullingTime="2.736716923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:13:14.410357044 +0000 UTC m=+423.721738315" watchObservedRunningTime="2026-07-15 04:13:14.411942725 +0000 UTC m=+423.723323983"
Jul 15 04:13:16.224472 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:16.224436    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:13:16.224744 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:16.224498    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:13:16.361521 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:16.361480    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs" event={"ID":"b825790a-c79c-472d-9c56-ac3a248491bc","Type":"ContainerStarted","Data":"563401e0df2429ea28a80e1a937709cabb560687a72d8653b406796d3e329890"}
Jul 15 04:13:16.361716 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:16.361639    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:16.385243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:16.385193    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs" podStartSLOduration=1.092088527 podStartE2EDuration="4.385175821s" totalImagesPullingTime="3.293087294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:13:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:13:16.383113661 +0000 UTC m=+425.694494908" watchObservedRunningTime="2026-07-15 04:13:16.385175821 +0000 UTC m=+425.696557068"
Jul 15 04:13:17.367564 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:17.367536    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-95pfs"
Jul 15 04:13:25.357458 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:13:25.357426    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-c845c8db6-f69db"
Jul 15 04:14:11.901480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:11.901448    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9c55f886c-jz7wp"]
Jul 15 04:14:11.905044 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:11.905023    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:11.916143 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:11.916118    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9c55f886c-jz7wp"]
Jul 15 04:14:12.051094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.051058    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ede7204a-1a33-4f03-8cf3-01d5e7851923-console-serving-cert\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.051282 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.051107    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-service-ca\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.051282 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.051131    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ede7204a-1a33-4f03-8cf3-01d5e7851923-console-oauth-config\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.051282 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.051162    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-console-config\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.051282 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.051192    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-776kf\" (UniqueName: \"kubernetes.io/projected/ede7204a-1a33-4f03-8cf3-01d5e7851923-kube-api-access-776kf\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.051282 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.051259    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-oauth-serving-cert\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.051468 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.051306    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-trusted-ca-bundle\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.151862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.151764    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-trusted-ca-bundle\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.151862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.151811    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ede7204a-1a33-4f03-8cf3-01d5e7851923-console-serving-cert\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.151862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.151841    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-service-ca\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.151862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.151858    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ede7204a-1a33-4f03-8cf3-01d5e7851923-console-oauth-config\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.152171 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.151878    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-console-config\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.152171 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.151901    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-776kf\" (UniqueName: \"kubernetes.io/projected/ede7204a-1a33-4f03-8cf3-01d5e7851923-kube-api-access-776kf\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.152171 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.151949    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-oauth-serving-cert\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.152674 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.152650    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-oauth-serving-cert\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.152782 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.152711    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-console-config\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.152782 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.152750    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-trusted-ca-bundle\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.152853 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.152804    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ede7204a-1a33-4f03-8cf3-01d5e7851923-service-ca\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.154473 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.154443    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ede7204a-1a33-4f03-8cf3-01d5e7851923-console-oauth-config\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.154559 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.154531    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ede7204a-1a33-4f03-8cf3-01d5e7851923-console-serving-cert\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.159438 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.159422    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-776kf\" (UniqueName: \"kubernetes.io/projected/ede7204a-1a33-4f03-8cf3-01d5e7851923-kube-api-access-776kf\") pod \"console-9c55f886c-jz7wp\" (UID: \"ede7204a-1a33-4f03-8cf3-01d5e7851923\") " pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.215653 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.215622    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:12.346572 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.346550    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9c55f886c-jz7wp"]
Jul 15 04:14:12.348972 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:14:12.348947    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podede7204a_1a33_4f03_8cf3_01d5e7851923.slice/crio-e66ca37c543a6274ed13c2abe0ef561d27c03e523c665979936974f001c9acf2 WatchSource:0}: Error finding container e66ca37c543a6274ed13c2abe0ef561d27c03e523c665979936974f001c9acf2: Status 404 returned error can't find the container with id e66ca37c543a6274ed13c2abe0ef561d27c03e523c665979936974f001c9acf2
Jul 15 04:14:12.598574 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.598532    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c55f886c-jz7wp" event={"ID":"ede7204a-1a33-4f03-8cf3-01d5e7851923","Type":"ContainerStarted","Data":"7a4bdcc860e7ea3df0452488b16c42029198b6e41b95282a6f8e64503ed824f6"}
Jul 15 04:14:12.598784 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.598650    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c55f886c-jz7wp" event={"ID":"ede7204a-1a33-4f03-8cf3-01d5e7851923","Type":"ContainerStarted","Data":"e66ca37c543a6274ed13c2abe0ef561d27c03e523c665979936974f001c9acf2"}
Jul 15 04:14:12.616508 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:12.616460    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9c55f886c-jz7wp" podStartSLOduration=1.616445505 podStartE2EDuration="1.616445505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:14:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:14:12.61495698 +0000 UTC m=+481.926338288" watchObservedRunningTime="2026-07-15 04:14:12.616445505 +0000 UTC m=+481.927826751"
Jul 15 04:14:22.216125 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:22.216081    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:22.216558 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:22.216147    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:22.220917 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:22.220893    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:22.645620 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:22.645569    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9c55f886c-jz7wp"
Jul 15 04:14:22.692085 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:22.692056    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57c44979fd-98d6d"]
Jul 15 04:14:47.713063 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:47.713021    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57c44979fd-98d6d" podUID="4f65fd5f-3605-4023-ab0e-4d3badf2329c" containerName="console" containerID="cri-o://3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c" gracePeriod=15
Jul 15 04:14:47.970905 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:47.970848    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:14:48.060954 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.060920    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-oauth-serving-cert\") pod \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") "
Jul 15 04:14:48.060954 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.060958    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-config\") pod \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") "
Jul 15 04:14:48.061197 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.060985    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-serving-cert\") pod \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") "
Jul 15 04:14:48.061197 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.061013    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-oauth-config\") pod \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") "
Jul 15 04:14:48.061197 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.061142    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-service-ca\") pod \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") "
Jul 15 04:14:48.061379 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.061200    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-trusted-ca-bundle\") pod \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") "
Jul 15 04:14:48.061379 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.061273    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kfvx\" (UniqueName: \"kubernetes.io/projected/4f65fd5f-3605-4023-ab0e-4d3badf2329c-kube-api-access-2kfvx\") pod \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\" (UID: \"4f65fd5f-3605-4023-ab0e-4d3badf2329c\") "
Jul 15 04:14:48.061518 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.061496    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4f65fd5f-3605-4023-ab0e-4d3badf2329c" (UID: "4f65fd5f-3605-4023-ab0e-4d3badf2329c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:14:48.061587 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.061565    2524 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-oauth-serving-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:14:48.061693 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.061616    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-service-ca" (OuterVolumeSpecName: "service-ca") pod "4f65fd5f-3605-4023-ab0e-4d3badf2329c" (UID: "4f65fd5f-3605-4023-ab0e-4d3badf2329c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:14:48.061914 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.061660    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4f65fd5f-3605-4023-ab0e-4d3badf2329c" (UID: "4f65fd5f-3605-4023-ab0e-4d3badf2329c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:14:48.061914 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.061767    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-config" (OuterVolumeSpecName: "console-config") pod "4f65fd5f-3605-4023-ab0e-4d3badf2329c" (UID: "4f65fd5f-3605-4023-ab0e-4d3badf2329c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:14:48.063252 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.063229    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4f65fd5f-3605-4023-ab0e-4d3badf2329c" (UID: "4f65fd5f-3605-4023-ab0e-4d3badf2329c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:14:48.063342 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.063313    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4f65fd5f-3605-4023-ab0e-4d3badf2329c" (UID: "4f65fd5f-3605-4023-ab0e-4d3badf2329c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:14:48.063447 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.063424    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f65fd5f-3605-4023-ab0e-4d3badf2329c-kube-api-access-2kfvx" (OuterVolumeSpecName: "kube-api-access-2kfvx") pod "4f65fd5f-3605-4023-ab0e-4d3badf2329c" (UID: "4f65fd5f-3605-4023-ab0e-4d3badf2329c"). InnerVolumeSpecName "kube-api-access-2kfvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:14:48.162024 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.161985    2524 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:14:48.162024 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.162017    2524 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-serving-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:14:48.162218 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.162038    2524 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f65fd5f-3605-4023-ab0e-4d3badf2329c-console-oauth-config\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:14:48.162218 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.162047    2524 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:14:48.162218 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.162055    2524 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f65fd5f-3605-4023-ab0e-4d3badf2329c-trusted-ca-bundle\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:14:48.162218 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.162063    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kfvx\" (UniqueName: \"kubernetes.io/projected/4f65fd5f-3605-4023-ab0e-4d3badf2329c-kube-api-access-2kfvx\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:14:48.751925 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.751889    2524 generic.go:356] "Generic (PLEG): container finished" podID="4f65fd5f-3605-4023-ab0e-4d3badf2329c" containerID="3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c" exitCode=0
Jul 15 04:14:48.752285 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.751941    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c44979fd-98d6d"
Jul 15 04:14:48.752285 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.751977    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c44979fd-98d6d" event={"ID":"4f65fd5f-3605-4023-ab0e-4d3badf2329c","Type":"ContainerDied","Data":"3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c"}
Jul 15 04:14:48.752285 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.752034    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c44979fd-98d6d" event={"ID":"4f65fd5f-3605-4023-ab0e-4d3badf2329c","Type":"ContainerDied","Data":"0551f3389852f873d469923b86d8e7a3ff06db573c88d03e31f211390ff5f6a9"}
Jul 15 04:14:48.752285 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.752055    2524 scope.go:117] "RemoveContainer" containerID="3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c"
Jul 15 04:14:48.761436 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.761420    2524 scope.go:117] "RemoveContainer" containerID="3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c"
Jul 15 04:14:48.761711 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:14:48.761692    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c\": container with ID starting with 3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c not found: ID does not exist" containerID="3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c"
Jul 15 04:14:48.761798 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.761723    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c"} err="failed to get container status \"3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c\": rpc error: code = NotFound desc = could not find container \"3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c\": container with ID starting with 3054806ce220adb62b35c6b03919c9c15368fa5726b34a402fd0f4a86866b08c not found: ID does not exist"
Jul 15 04:14:48.773073 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.773046    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57c44979fd-98d6d"]
Jul 15 04:14:48.785289 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:48.785255    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57c44979fd-98d6d"]
Jul 15 04:14:49.431677 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:14:49.431640    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f65fd5f-3605-4023-ab0e-4d3badf2329c" path="/var/lib/kubelet/pods/4f65fd5f-3605-4023-ab0e-4d3badf2329c/volumes"
Jul 15 04:15:09.794744 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.794657    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"]
Jul 15 04:15:09.795198 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.794960    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f65fd5f-3605-4023-ab0e-4d3badf2329c" containerName="console"
Jul 15 04:15:09.795198 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.794970    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f65fd5f-3605-4023-ab0e-4d3badf2329c" containerName="console"
Jul 15 04:15:09.795198 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.795031    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f65fd5f-3605-4023-ab0e-4d3badf2329c" containerName="console"
Jul 15 04:15:09.798103 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.798080    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:09.802521 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.802499    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d79pv\""
Jul 15 04:15:09.803031 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.803014    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 04:15:09.803122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.803060    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 04:15:09.815663 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.815632    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"]
Jul 15 04:15:09.863091 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.863062    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b69d02f-8155-467f-a188-573118546d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6fqjz\" (UID: \"2b69d02f-8155-467f-a188-573118546d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:09.863250 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.863108    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww8h7\" (UniqueName: \"kubernetes.io/projected/2b69d02f-8155-467f-a188-573118546d6c-kube-api-access-ww8h7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6fqjz\" (UID: \"2b69d02f-8155-467f-a188-573118546d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:09.964474 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.964419    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b69d02f-8155-467f-a188-573118546d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6fqjz\" (UID: \"2b69d02f-8155-467f-a188-573118546d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:09.964683 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.964518    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww8h7\" (UniqueName: \"kubernetes.io/projected/2b69d02f-8155-467f-a188-573118546d6c-kube-api-access-ww8h7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6fqjz\" (UID: \"2b69d02f-8155-467f-a188-573118546d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:09.964922 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.964897    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b69d02f-8155-467f-a188-573118546d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6fqjz\" (UID: \"2b69d02f-8155-467f-a188-573118546d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:09.974112 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:09.974090    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww8h7\" (UniqueName: \"kubernetes.io/projected/2b69d02f-8155-467f-a188-573118546d6c-kube-api-access-ww8h7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6fqjz\" (UID: \"2b69d02f-8155-467f-a188-573118546d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:10.108545 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:10.108458    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:10.242306 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:10.242282    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"]
Jul 15 04:15:10.244934 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:15:10.244906    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b69d02f_8155_467f_a188_573118546d6c.slice/crio-a2285c0f77e5cb5840344172c1336e48435e593ab5c1bde515c16d15d6443982 WatchSource:0}: Error finding container a2285c0f77e5cb5840344172c1336e48435e593ab5c1bde515c16d15d6443982: Status 404 returned error can't find the container with id a2285c0f77e5cb5840344172c1336e48435e593ab5c1bde515c16d15d6443982
Jul 15 04:15:10.588133 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:10.588088    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"]
Jul 15 04:15:10.592228 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:10.592194    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"]
Jul 15 04:15:14.870660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:14.870619    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz" podUID="2b69d02f-8155-467f-a188-573118546d6c" containerName="manager" containerID="cri-o://773770c80c6c2f7072897e0f68c54c4bf7a026b72ddde92d50e4be3507f0236e" gracePeriod=2
Jul 15 04:15:15.116324 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.116295    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:15.122316 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:15:15.122233    2524 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6fqjz\" is forbidden: User \"system:node:ip-10-0-129-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-76.ec2.internal' and this object" podUID="2b69d02f-8155-467f-a188-573118546d6c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:15.212419 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.212380    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww8h7\" (UniqueName: \"kubernetes.io/projected/2b69d02f-8155-467f-a188-573118546d6c-kube-api-access-ww8h7\") pod \"2b69d02f-8155-467f-a188-573118546d6c\" (UID: \"2b69d02f-8155-467f-a188-573118546d6c\") "
Jul 15 04:15:15.212644 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.212449    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b69d02f-8155-467f-a188-573118546d6c-extensions-socket-volume\") pod \"2b69d02f-8155-467f-a188-573118546d6c\" (UID: \"2b69d02f-8155-467f-a188-573118546d6c\") "
Jul 15 04:15:15.212809 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.212779    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b69d02f-8155-467f-a188-573118546d6c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2b69d02f-8155-467f-a188-573118546d6c" (UID: "2b69d02f-8155-467f-a188-573118546d6c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:15:15.214736 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.214711    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b69d02f-8155-467f-a188-573118546d6c-kube-api-access-ww8h7" (OuterVolumeSpecName: "kube-api-access-ww8h7") pod "2b69d02f-8155-467f-a188-573118546d6c" (UID: "2b69d02f-8155-467f-a188-573118546d6c"). InnerVolumeSpecName "kube-api-access-ww8h7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:15:15.313458 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.313420    2524 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b69d02f-8155-467f-a188-573118546d6c-extensions-socket-volume\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:15:15.313458 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.313452    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww8h7\" (UniqueName: \"kubernetes.io/projected/2b69d02f-8155-467f-a188-573118546d6c-kube-api-access-ww8h7\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:15:15.431492 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.431415    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b69d02f-8155-467f-a188-573118546d6c" path="/var/lib/kubelet/pods/2b69d02f-8155-467f-a188-573118546d6c/volumes"
Jul 15 04:15:15.876310 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.876277    2524 generic.go:356] "Generic (PLEG): container finished" podID="2b69d02f-8155-467f-a188-573118546d6c" containerID="773770c80c6c2f7072897e0f68c54c4bf7a026b72ddde92d50e4be3507f0236e" exitCode=2
Jul 15 04:15:15.876786 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.876321    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:15.876786 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.876348    2524 scope.go:117] "RemoveContainer" containerID="773770c80c6c2f7072897e0f68c54c4bf7a026b72ddde92d50e4be3507f0236e"
Jul 15 04:15:15.881150 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:15:15.881121    2524 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6fqjz\" is forbidden: User \"system:node:ip-10-0-129-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-76.ec2.internal' and this object" podUID="2b69d02f-8155-467f-a188-573118546d6c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6fqjz"
Jul 15 04:15:15.886870 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.886847    2524 scope.go:117] "RemoveContainer" containerID="773770c80c6c2f7072897e0f68c54c4bf7a026b72ddde92d50e4be3507f0236e"
Jul 15 04:15:15.887126 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:15:15.887105    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"773770c80c6c2f7072897e0f68c54c4bf7a026b72ddde92d50e4be3507f0236e\": container with ID starting with 773770c80c6c2f7072897e0f68c54c4bf7a026b72ddde92d50e4be3507f0236e not found: ID does not exist" containerID="773770c80c6c2f7072897e0f68c54c4bf7a026b72ddde92d50e4be3507f0236e"
Jul 15 04:15:15.887182 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:15.887134    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"773770c80c6c2f7072897e0f68c54c4bf7a026b72ddde92d50e4be3507f0236e"} err="failed to get container status \"773770c80c6c2f7072897e0f68c54c4bf7a026b72ddde92d50e4be3507f0236e\": rpc error: code = NotFound desc = could not find container \"773770c80c6c2f7072897e0f68c54c4bf7a026b72ddde92d50e4be3507f0236e\": container with ID starting with 773770c80c6c2f7072897e0f68c54c4bf7a026b72ddde92d50e4be3507f0236e not found: ID does not exist"
Jul 15 04:15:53.978436 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:53.978390    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"]
Jul 15 04:15:53.978999 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:53.978852    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b69d02f-8155-467f-a188-573118546d6c" containerName="manager"
Jul 15 04:15:53.978999 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:53.978871    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b69d02f-8155-467f-a188-573118546d6c" containerName="manager"
Jul 15 04:15:53.978999 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:53.978970    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b69d02f-8155-467f-a188-573118546d6c" containerName="manager"
Jul 15 04:15:53.984079 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:53.984046    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:15:53.986963 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:53.986935    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qzv8h\""
Jul 15 04:15:53.987123 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:53.987033    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 15 04:15:53.987866 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:53.987844    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 04:15:53.987984 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:53.987848    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 04:15:53.991264 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:53.991235    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"]
Jul 15 04:15:54.059534 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:54.059497    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a10c1ccf-d683-4e22-b328-541aac904dc0-config-file\") pod \"limitador-limitador-64c94fd64f-bv5gd\" (UID: \"a10c1ccf-d683-4e22-b328-541aac904dc0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:15:54.059534 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:54.059543    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcvdv\" (UniqueName: \"kubernetes.io/projected/a10c1ccf-d683-4e22-b328-541aac904dc0-kube-api-access-gcvdv\") pod \"limitador-limitador-64c94fd64f-bv5gd\" (UID: \"a10c1ccf-d683-4e22-b328-541aac904dc0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:15:54.077220 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:54.077184    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"]
Jul 15 04:15:54.160735 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:54.160694    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a10c1ccf-d683-4e22-b328-541aac904dc0-config-file\") pod \"limitador-limitador-64c94fd64f-bv5gd\" (UID: \"a10c1ccf-d683-4e22-b328-541aac904dc0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:15:54.160735 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:54.160735    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcvdv\" (UniqueName: \"kubernetes.io/projected/a10c1ccf-d683-4e22-b328-541aac904dc0-kube-api-access-gcvdv\") pod \"limitador-limitador-64c94fd64f-bv5gd\" (UID: \"a10c1ccf-d683-4e22-b328-541aac904dc0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:15:54.161328 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:54.161307    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a10c1ccf-d683-4e22-b328-541aac904dc0-config-file\") pod \"limitador-limitador-64c94fd64f-bv5gd\" (UID: \"a10c1ccf-d683-4e22-b328-541aac904dc0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:15:54.172472 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:54.172437    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcvdv\" (UniqueName: \"kubernetes.io/projected/a10c1ccf-d683-4e22-b328-541aac904dc0-kube-api-access-gcvdv\") pod \"limitador-limitador-64c94fd64f-bv5gd\" (UID: \"a10c1ccf-d683-4e22-b328-541aac904dc0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:15:54.296900 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:54.296829    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:15:54.424025 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:54.423992    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"]
Jul 15 04:15:54.427856 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:15:54.427830    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda10c1ccf_d683_4e22_b328_541aac904dc0.slice/crio-87ec7fdec6efa8f7f37b1c9633eec6fd7570f8f0215cf9410b800a16d4e0e431 WatchSource:0}: Error finding container 87ec7fdec6efa8f7f37b1c9633eec6fd7570f8f0215cf9410b800a16d4e0e431: Status 404 returned error can't find the container with id 87ec7fdec6efa8f7f37b1c9633eec6fd7570f8f0215cf9410b800a16d4e0e431
Jul 15 04:15:55.055080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:55.055040    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd" event={"ID":"a10c1ccf-d683-4e22-b328-541aac904dc0","Type":"ContainerStarted","Data":"87ec7fdec6efa8f7f37b1c9633eec6fd7570f8f0215cf9410b800a16d4e0e431"}
Jul 15 04:15:58.070186 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:58.070148    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd" event={"ID":"a10c1ccf-d683-4e22-b328-541aac904dc0","Type":"ContainerStarted","Data":"ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a"}
Jul 15 04:15:58.070566 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:58.070223    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:15:58.088073 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:15:58.088029    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd" podStartSLOduration=2.339259108 podStartE2EDuration="5.088013347s" totalImagesPullingTime="2.748754239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:15:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:15:58.087294028 +0000 UTC m=+587.398675300" watchObservedRunningTime="2026-07-15 04:15:58.088013347 +0000 UTC m=+587.399394619"
Jul 15 04:16:09.075814 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:09.075782    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:16:09.175892 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:09.175855    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"]
Jul 15 04:16:09.176132 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:09.176092    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd" podUID="a10c1ccf-d683-4e22-b328-541aac904dc0" containerName="limitador" containerID="cri-o://ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a" gracePeriod=30
Jul 15 04:16:09.719482 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:09.719460    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:16:09.792644 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:09.792542    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcvdv\" (UniqueName: \"kubernetes.io/projected/a10c1ccf-d683-4e22-b328-541aac904dc0-kube-api-access-gcvdv\") pod \"a10c1ccf-d683-4e22-b328-541aac904dc0\" (UID: \"a10c1ccf-d683-4e22-b328-541aac904dc0\") "
Jul 15 04:16:09.792644 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:09.792628    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a10c1ccf-d683-4e22-b328-541aac904dc0-config-file\") pod \"a10c1ccf-d683-4e22-b328-541aac904dc0\" (UID: \"a10c1ccf-d683-4e22-b328-541aac904dc0\") "
Jul 15 04:16:09.792987 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:09.792962    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a10c1ccf-d683-4e22-b328-541aac904dc0-config-file" (OuterVolumeSpecName: "config-file") pod "a10c1ccf-d683-4e22-b328-541aac904dc0" (UID: "a10c1ccf-d683-4e22-b328-541aac904dc0"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:16:09.794785 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:09.794764    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a10c1ccf-d683-4e22-b328-541aac904dc0-kube-api-access-gcvdv" (OuterVolumeSpecName: "kube-api-access-gcvdv") pod "a10c1ccf-d683-4e22-b328-541aac904dc0" (UID: "a10c1ccf-d683-4e22-b328-541aac904dc0"). InnerVolumeSpecName "kube-api-access-gcvdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:16:09.893168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:09.893126    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcvdv\" (UniqueName: \"kubernetes.io/projected/a10c1ccf-d683-4e22-b328-541aac904dc0-kube-api-access-gcvdv\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:16:09.893168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:09.893159    2524 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a10c1ccf-d683-4e22-b328-541aac904dc0-config-file\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:16:10.129585 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:10.129547    2524 generic.go:356] "Generic (PLEG): container finished" podID="a10c1ccf-d683-4e22-b328-541aac904dc0" containerID="ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a" exitCode=0
Jul 15 04:16:10.130004 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:10.129612    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"
Jul 15 04:16:10.130004 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:10.129640    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd" event={"ID":"a10c1ccf-d683-4e22-b328-541aac904dc0","Type":"ContainerDied","Data":"ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a"}
Jul 15 04:16:10.130004 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:10.129699    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bv5gd" event={"ID":"a10c1ccf-d683-4e22-b328-541aac904dc0","Type":"ContainerDied","Data":"87ec7fdec6efa8f7f37b1c9633eec6fd7570f8f0215cf9410b800a16d4e0e431"}
Jul 15 04:16:10.130004 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:10.129721    2524 scope.go:117] "RemoveContainer" containerID="ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a"
Jul 15 04:16:10.138222 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:10.138206    2524 scope.go:117] "RemoveContainer" containerID="ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a"
Jul 15 04:16:10.138499 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:16:10.138482    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a\": container with ID starting with ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a not found: ID does not exist" containerID="ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a"
Jul 15 04:16:10.138549 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:10.138510    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a"} err="failed to get container status \"ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a\": rpc error: code = NotFound desc = could not find container \"ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a\": container with ID starting with ae98b646b6f1b1870440cd1103f803fe093afc1b29e350a0b3bb7f9c0eb2835a not found: ID does not exist"
Jul 15 04:16:10.149512 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:10.149478    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"]
Jul 15 04:16:10.153237 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:10.153216    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bv5gd"]
Jul 15 04:16:11.297482 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:11.297453    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:16:11.299677 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:11.299654    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:16:11.432055 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:11.432024    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a10c1ccf-d683-4e22-b328-541aac904dc0" path="/var/lib/kubelet/pods/a10c1ccf-d683-4e22-b328-541aac904dc0/volumes"
Jul 15 04:16:13.423583 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.423540    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"]
Jul 15 04:16:13.424073 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.424052    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a10c1ccf-d683-4e22-b328-541aac904dc0" containerName="limitador"
Jul 15 04:16:13.424163 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.424076    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="a10c1ccf-d683-4e22-b328-541aac904dc0" containerName="limitador"
Jul 15 04:16:13.424216 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.424200    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="a10c1ccf-d683-4e22-b328-541aac904dc0" containerName="limitador"
Jul 15 04:16:13.427013 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.426981    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"
Jul 15 04:16:13.431877 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.431856    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zldfr\""
Jul 15 04:16:13.431997 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.431882    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 15 04:16:13.431997 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.431858    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 04:16:13.432787 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.432769    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 04:16:13.440191 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.440170    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"]
Jul 15 04:16:13.522990 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.522948    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7x8c\" (UniqueName: \"kubernetes.io/projected/33f2aa1e-a28c-41aa-9671-780931d50bbe-kube-api-access-c7x8c\") pod \"postgres-69fb4594cf-m8pr4\" (UID: \"33f2aa1e-a28c-41aa-9671-780931d50bbe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"
Jul 15 04:16:13.523156 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.523013    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/33f2aa1e-a28c-41aa-9671-780931d50bbe-data\") pod \"postgres-69fb4594cf-m8pr4\" (UID: \"33f2aa1e-a28c-41aa-9671-780931d50bbe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"
Jul 15 04:16:13.623466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.623436    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/33f2aa1e-a28c-41aa-9671-780931d50bbe-data\") pod \"postgres-69fb4594cf-m8pr4\" (UID: \"33f2aa1e-a28c-41aa-9671-780931d50bbe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"
Jul 15 04:16:13.623705 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.623521    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7x8c\" (UniqueName: \"kubernetes.io/projected/33f2aa1e-a28c-41aa-9671-780931d50bbe-kube-api-access-c7x8c\") pod \"postgres-69fb4594cf-m8pr4\" (UID: \"33f2aa1e-a28c-41aa-9671-780931d50bbe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"
Jul 15 04:16:13.623836 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.623814    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/33f2aa1e-a28c-41aa-9671-780931d50bbe-data\") pod \"postgres-69fb4594cf-m8pr4\" (UID: \"33f2aa1e-a28c-41aa-9671-780931d50bbe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"
Jul 15 04:16:13.631428 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.631403    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7x8c\" (UniqueName: \"kubernetes.io/projected/33f2aa1e-a28c-41aa-9671-780931d50bbe-kube-api-access-c7x8c\") pod \"postgres-69fb4594cf-m8pr4\" (UID: \"33f2aa1e-a28c-41aa-9671-780931d50bbe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"
Jul 15 04:16:13.737485 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.737405    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"
Jul 15 04:16:13.863233 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:13.863210    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"]
Jul 15 04:16:13.865935 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:16:13.865903    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f2aa1e_a28c_41aa_9671_780931d50bbe.slice/crio-d86b6211f23715bd745d046fe320f2f982c5c3d1babc332ea120eb6f9c73be02 WatchSource:0}: Error finding container d86b6211f23715bd745d046fe320f2f982c5c3d1babc332ea120eb6f9c73be02: Status 404 returned error can't find the container with id d86b6211f23715bd745d046fe320f2f982c5c3d1babc332ea120eb6f9c73be02
Jul 15 04:16:14.148433 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:14.148401    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4" event={"ID":"33f2aa1e-a28c-41aa-9671-780931d50bbe","Type":"ContainerStarted","Data":"d86b6211f23715bd745d046fe320f2f982c5c3d1babc332ea120eb6f9c73be02"}
Jul 15 04:16:19.177456 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:19.177367    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4" event={"ID":"33f2aa1e-a28c-41aa-9671-780931d50bbe","Type":"ContainerStarted","Data":"4046049f6b4d23c3030809815b30128d1e5312d3f407ad9392c6ded1d1c04792"}
Jul 15 04:16:19.177456 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:19.177443    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"
Jul 15 04:16:25.211974 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:25.211944    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4"
Jul 15 04:16:25.228102 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:25.228052    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m8pr4" podStartSLOduration=7.219997101 podStartE2EDuration="12.228033882s" totalImagesPullingTime="5.008036781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:16:19.194089507 +0000 UTC m=+608.505470754" watchObservedRunningTime="2026-07-15 04:16:25.228033882 +0000 UTC m=+614.539415127"
Jul 15 04:16:32.547684 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.547648    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-54dfbc96fc-x7f95"]
Jul 15 04:16:32.730937 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.730856    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54dfbc96fc-x7f95"]
Jul 15 04:16:32.731081 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.730984    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:32.734791 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.734768    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 04:16:32.734919 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.734770    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 04:16:32.734919 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.734770    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dwf88\""
Jul 15 04:16:32.740482 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.740461    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67d859fc9c-g9ll2"]
Jul 15 04:16:32.758119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.758098    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67d859fc9c-g9ll2"]
Jul 15 04:16:32.758208 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.758183    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:16:32.891352 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.891317    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfhcr\" (UniqueName: \"kubernetes.io/projected/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-kube-api-access-lfhcr\") pod \"maas-controller-54dfbc96fc-x7f95\" (UID: \"b11977ec-c188-4d4c-8280-8a78f8b9fe3d\") " pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:32.891514 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.891358    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg2x5\" (UniqueName: \"kubernetes.io/projected/230b3b5d-8621-4c1f-8f39-52e467afdefe-kube-api-access-lg2x5\") pod \"maas-controller-67d859fc9c-g9ll2\" (UID: \"230b3b5d-8621-4c1f-8f39-52e467afdefe\") " pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:16:32.891514 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.891381    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/230b3b5d-8621-4c1f-8f39-52e467afdefe-webhook-cert\") pod \"maas-controller-67d859fc9c-g9ll2\" (UID: \"230b3b5d-8621-4c1f-8f39-52e467afdefe\") " pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:16:32.891514 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.891407    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-webhook-cert\") pod \"maas-controller-54dfbc96fc-x7f95\" (UID: \"b11977ec-c188-4d4c-8280-8a78f8b9fe3d\") " pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:32.991890 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.991858    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfhcr\" (UniqueName: \"kubernetes.io/projected/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-kube-api-access-lfhcr\") pod \"maas-controller-54dfbc96fc-x7f95\" (UID: \"b11977ec-c188-4d4c-8280-8a78f8b9fe3d\") " pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:32.992080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.991897    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg2x5\" (UniqueName: \"kubernetes.io/projected/230b3b5d-8621-4c1f-8f39-52e467afdefe-kube-api-access-lg2x5\") pod \"maas-controller-67d859fc9c-g9ll2\" (UID: \"230b3b5d-8621-4c1f-8f39-52e467afdefe\") " pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:16:32.992080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.991936    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/230b3b5d-8621-4c1f-8f39-52e467afdefe-webhook-cert\") pod \"maas-controller-67d859fc9c-g9ll2\" (UID: \"230b3b5d-8621-4c1f-8f39-52e467afdefe\") " pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:16:32.992080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.991959    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-webhook-cert\") pod \"maas-controller-54dfbc96fc-x7f95\" (UID: \"b11977ec-c188-4d4c-8280-8a78f8b9fe3d\") " pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:32.994621 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.994569    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/230b3b5d-8621-4c1f-8f39-52e467afdefe-webhook-cert\") pod \"maas-controller-67d859fc9c-g9ll2\" (UID: \"230b3b5d-8621-4c1f-8f39-52e467afdefe\") " pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:16:32.994815 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.994795    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-webhook-cert\") pod \"maas-controller-54dfbc96fc-x7f95\" (UID: \"b11977ec-c188-4d4c-8280-8a78f8b9fe3d\") " pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:32.999689 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.999669    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg2x5\" (UniqueName: \"kubernetes.io/projected/230b3b5d-8621-4c1f-8f39-52e467afdefe-kube-api-access-lg2x5\") pod \"maas-controller-67d859fc9c-g9ll2\" (UID: \"230b3b5d-8621-4c1f-8f39-52e467afdefe\") " pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:16:32.999861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:32.999766    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfhcr\" (UniqueName: \"kubernetes.io/projected/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-kube-api-access-lfhcr\") pod \"maas-controller-54dfbc96fc-x7f95\" (UID: \"b11977ec-c188-4d4c-8280-8a78f8b9fe3d\") " pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:33.041007 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:33.040976    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:33.071214 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:33.071184    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:16:33.215476 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:33.215449    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54dfbc96fc-x7f95"]
Jul 15 04:16:33.217949 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:16:33.217918    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb11977ec_c188_4d4c_8280_8a78f8b9fe3d.slice/crio-f20d9233e6fef8a8bfa5d09f74fa2c47fb3e44acf2245bcfdb14c5acd8976b50 WatchSource:0}: Error finding container f20d9233e6fef8a8bfa5d09f74fa2c47fb3e44acf2245bcfdb14c5acd8976b50: Status 404 returned error can't find the container with id f20d9233e6fef8a8bfa5d09f74fa2c47fb3e44acf2245bcfdb14c5acd8976b50
Jul 15 04:16:33.238350 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:33.238315    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54dfbc96fc-x7f95" event={"ID":"b11977ec-c188-4d4c-8280-8a78f8b9fe3d","Type":"ContainerStarted","Data":"f20d9233e6fef8a8bfa5d09f74fa2c47fb3e44acf2245bcfdb14c5acd8976b50"}
Jul 15 04:16:33.239428 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:33.239411    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67d859fc9c-g9ll2"]
Jul 15 04:16:33.241242 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:16:33.241219    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230b3b5d_8621_4c1f_8f39_52e467afdefe.slice/crio-dee4ac5a1b7522c8c7d16f355d62eefd36420f0817c282d628c4cc3dbd7807a0 WatchSource:0}: Error finding container dee4ac5a1b7522c8c7d16f355d62eefd36420f0817c282d628c4cc3dbd7807a0: Status 404 returned error can't find the container with id dee4ac5a1b7522c8c7d16f355d62eefd36420f0817c282d628c4cc3dbd7807a0
Jul 15 04:16:34.247535 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:34.247458    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67d859fc9c-g9ll2" event={"ID":"230b3b5d-8621-4c1f-8f39-52e467afdefe","Type":"ContainerStarted","Data":"dee4ac5a1b7522c8c7d16f355d62eefd36420f0817c282d628c4cc3dbd7807a0"}
Jul 15 04:16:36.258577 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:36.258531    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67d859fc9c-g9ll2" event={"ID":"230b3b5d-8621-4c1f-8f39-52e467afdefe","Type":"ContainerStarted","Data":"8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1"}
Jul 15 04:16:36.259086 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:36.258668    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:16:36.260115 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:36.260091    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54dfbc96fc-x7f95" event={"ID":"b11977ec-c188-4d4c-8280-8a78f8b9fe3d","Type":"ContainerStarted","Data":"4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be"}
Jul 15 04:16:36.260184 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:36.260150    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:36.278700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:36.278650    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67d859fc9c-g9ll2" podStartSLOduration=1.676691308 podStartE2EDuration="4.278637583s" totalImagesPullingTime="2.601946275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:16:36.276358192 +0000 UTC m=+625.587739465" watchObservedRunningTime="2026-07-15 04:16:36.278637583 +0000 UTC m=+625.590018829"
Jul 15 04:16:36.292359 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:36.292312    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-54dfbc96fc-x7f95" podStartSLOduration=1.658383819 podStartE2EDuration="4.29229806s" totalImagesPullingTime="2.633914241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:16:36.291257205 +0000 UTC m=+625.602638453" watchObservedRunningTime="2026-07-15 04:16:36.29229806 +0000 UTC m=+625.603679305"
Jul 15 04:16:47.267511 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:47.267479    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:16:47.268086 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:47.267555    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:47.331209 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:47.331176    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54dfbc96fc-x7f95"]
Jul 15 04:16:47.331422 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:47.331393    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-54dfbc96fc-x7f95" podUID="b11977ec-c188-4d4c-8280-8a78f8b9fe3d" containerName="manager" containerID="cri-o://4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be" gracePeriod=30
Jul 15 04:16:47.582606 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:47.582570    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:47.717482 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:47.717437    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfhcr\" (UniqueName: \"kubernetes.io/projected/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-kube-api-access-lfhcr\") pod \"b11977ec-c188-4d4c-8280-8a78f8b9fe3d\" (UID: \"b11977ec-c188-4d4c-8280-8a78f8b9fe3d\") "
Jul 15 04:16:47.717718 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:47.717539    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-webhook-cert\") pod \"b11977ec-c188-4d4c-8280-8a78f8b9fe3d\" (UID: \"b11977ec-c188-4d4c-8280-8a78f8b9fe3d\") "
Jul 15 04:16:47.720035 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:47.719998    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b11977ec-c188-4d4c-8280-8a78f8b9fe3d" (UID: "b11977ec-c188-4d4c-8280-8a78f8b9fe3d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:16:47.720167 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:47.719995    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-kube-api-access-lfhcr" (OuterVolumeSpecName: "kube-api-access-lfhcr") pod "b11977ec-c188-4d4c-8280-8a78f8b9fe3d" (UID: "b11977ec-c188-4d4c-8280-8a78f8b9fe3d"). InnerVolumeSpecName "kube-api-access-lfhcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:16:47.818462 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:47.818376    2524 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-webhook-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:16:47.818462 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:47.818407    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfhcr\" (UniqueName: \"kubernetes.io/projected/b11977ec-c188-4d4c-8280-8a78f8b9fe3d-kube-api-access-lfhcr\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:16:48.314280 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:48.314244    2524 generic.go:356] "Generic (PLEG): container finished" podID="b11977ec-c188-4d4c-8280-8a78f8b9fe3d" containerID="4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be" exitCode=0
Jul 15 04:16:48.314700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:48.314295    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54dfbc96fc-x7f95"
Jul 15 04:16:48.314700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:48.314331    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54dfbc96fc-x7f95" event={"ID":"b11977ec-c188-4d4c-8280-8a78f8b9fe3d","Type":"ContainerDied","Data":"4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be"}
Jul 15 04:16:48.314700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:48.314388    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54dfbc96fc-x7f95" event={"ID":"b11977ec-c188-4d4c-8280-8a78f8b9fe3d","Type":"ContainerDied","Data":"f20d9233e6fef8a8bfa5d09f74fa2c47fb3e44acf2245bcfdb14c5acd8976b50"}
Jul 15 04:16:48.314700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:48.314416    2524 scope.go:117] "RemoveContainer" containerID="4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be"
Jul 15 04:16:48.322971 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:48.322955    2524 scope.go:117] "RemoveContainer" containerID="4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be"
Jul 15 04:16:48.323214 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:16:48.323199    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be\": container with ID starting with 4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be not found: ID does not exist" containerID="4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be"
Jul 15 04:16:48.323268 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:48.323221    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be"} err="failed to get container status \"4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be\": rpc error: code = NotFound desc = could not find container \"4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be\": container with ID starting with 4ad434fab210a4677e96ca09aed577530df43533c42e68902055abfbb63b49be not found: ID does not exist"
Jul 15 04:16:48.335962 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:48.335933    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54dfbc96fc-x7f95"]
Jul 15 04:16:48.339998 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:48.339978    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-54dfbc96fc-x7f95"]
Jul 15 04:16:49.432344 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:16:49.432312    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b11977ec-c188-4d4c-8280-8a78f8b9fe3d" path="/var/lib/kubelet/pods/b11977ec-c188-4d4c-8280-8a78f8b9fe3d/volumes"
Jul 15 04:17:07.626548 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.626511    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"]
Jul 15 04:17:07.626991 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.626917    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b11977ec-c188-4d4c-8280-8a78f8b9fe3d" containerName="manager"
Jul 15 04:17:07.626991 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.626930    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="b11977ec-c188-4d4c-8280-8a78f8b9fe3d" containerName="manager"
Jul 15 04:17:07.626991 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.626984    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="b11977ec-c188-4d4c-8280-8a78f8b9fe3d" containerName="manager"
Jul 15 04:17:07.629561 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.629533    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:07.632094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.632067    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 15 04:17:07.632271 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.632250    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sg5pd\""
Jul 15 04:17:07.638263 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.638238    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"]
Jul 15 04:17:07.699213 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.699175    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m85qr\" (UniqueName: \"kubernetes.io/projected/657ae513-60d6-471a-8bb6-7d2b2e27dd32-kube-api-access-m85qr\") pod \"maas-api-664cd857c8-qv8rg\" (UID: \"657ae513-60d6-471a-8bb6-7d2b2e27dd32\") " pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:07.699403 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.699230    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/657ae513-60d6-471a-8bb6-7d2b2e27dd32-maas-api-tls\") pod \"maas-api-664cd857c8-qv8rg\" (UID: \"657ae513-60d6-471a-8bb6-7d2b2e27dd32\") " pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:07.699403 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.699286    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/657ae513-60d6-471a-8bb6-7d2b2e27dd32-openshift-service-ca\") pod \"maas-api-664cd857c8-qv8rg\" (UID: \"657ae513-60d6-471a-8bb6-7d2b2e27dd32\") " pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:07.800506 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.800463    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m85qr\" (UniqueName: \"kubernetes.io/projected/657ae513-60d6-471a-8bb6-7d2b2e27dd32-kube-api-access-m85qr\") pod \"maas-api-664cd857c8-qv8rg\" (UID: \"657ae513-60d6-471a-8bb6-7d2b2e27dd32\") " pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:07.800704 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.800529    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/657ae513-60d6-471a-8bb6-7d2b2e27dd32-maas-api-tls\") pod \"maas-api-664cd857c8-qv8rg\" (UID: \"657ae513-60d6-471a-8bb6-7d2b2e27dd32\") " pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:07.800704 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.800563    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/657ae513-60d6-471a-8bb6-7d2b2e27dd32-openshift-service-ca\") pod \"maas-api-664cd857c8-qv8rg\" (UID: \"657ae513-60d6-471a-8bb6-7d2b2e27dd32\") " pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:07.800778 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:17:07.800707    2524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 15 04:17:07.800816 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:17:07.800795    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657ae513-60d6-471a-8bb6-7d2b2e27dd32-maas-api-tls podName:657ae513-60d6-471a-8bb6-7d2b2e27dd32 nodeName:}" failed. No retries permitted until 2026-07-15 04:17:08.300771545 +0000 UTC m=+657.612152783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/657ae513-60d6-471a-8bb6-7d2b2e27dd32-maas-api-tls") pod "maas-api-664cd857c8-qv8rg" (UID: "657ae513-60d6-471a-8bb6-7d2b2e27dd32") : secret "maas-api-serving-cert" not found
Jul 15 04:17:07.801183 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.801166    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/657ae513-60d6-471a-8bb6-7d2b2e27dd32-openshift-service-ca\") pod \"maas-api-664cd857c8-qv8rg\" (UID: \"657ae513-60d6-471a-8bb6-7d2b2e27dd32\") " pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:07.811321 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:07.811293    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m85qr\" (UniqueName: \"kubernetes.io/projected/657ae513-60d6-471a-8bb6-7d2b2e27dd32-kube-api-access-m85qr\") pod \"maas-api-664cd857c8-qv8rg\" (UID: \"657ae513-60d6-471a-8bb6-7d2b2e27dd32\") " pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:08.151880 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.151851    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"]
Jul 15 04:17:08.155448 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.155427    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"
Jul 15 04:17:08.158420 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.158399    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 15 04:17:08.158511 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.158431    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rlp9q\""
Jul 15 04:17:08.167380 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.167353    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"]
Jul 15 04:17:08.171078 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.171059    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"]
Jul 15 04:17:08.171173 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.171157    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"
Jul 15 04:17:08.184052 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.184028    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"]
Jul 15 04:17:08.204441 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.204410    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n42np\" (UniqueName: \"kubernetes.io/projected/adefd0a4-cd6b-4b91-9aa9-da45df2ca089-kube-api-access-n42np\") pod \"payload-pre-processing-74584bb9f8-7b9nq\" (UID: \"adefd0a4-cd6b-4b91-9aa9-da45df2ca089\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"
Jul 15 04:17:08.204574 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.204482    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adefd0a4-cd6b-4b91-9aa9-da45df2ca089-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-7b9nq\" (UID: \"adefd0a4-cd6b-4b91-9aa9-da45df2ca089\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"
Jul 15 04:17:08.305543 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.305503    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n42np\" (UniqueName: \"kubernetes.io/projected/adefd0a4-cd6b-4b91-9aa9-da45df2ca089-kube-api-access-n42np\") pod \"payload-pre-processing-74584bb9f8-7b9nq\" (UID: \"adefd0a4-cd6b-4b91-9aa9-da45df2ca089\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"
Jul 15 04:17:08.305741 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.305559    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkcdv\" (UniqueName: \"kubernetes.io/projected/809162ae-0b69-4f2d-ba1a-7a97ec49f18c-kube-api-access-nkcdv\") pod \"payload-processing-f9b8d9b9b-wrt6d\" (UID: \"809162ae-0b69-4f2d-ba1a-7a97ec49f18c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"
Jul 15 04:17:08.305741 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.305633    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/657ae513-60d6-471a-8bb6-7d2b2e27dd32-maas-api-tls\") pod \"maas-api-664cd857c8-qv8rg\" (UID: \"657ae513-60d6-471a-8bb6-7d2b2e27dd32\") " pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:08.305817 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.305746    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adefd0a4-cd6b-4b91-9aa9-da45df2ca089-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-7b9nq\" (UID: \"adefd0a4-cd6b-4b91-9aa9-da45df2ca089\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"
Jul 15 04:17:08.305817 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.305796    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/809162ae-0b69-4f2d-ba1a-7a97ec49f18c-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-wrt6d\" (UID: \"809162ae-0b69-4f2d-ba1a-7a97ec49f18c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"
Jul 15 04:17:08.306508 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.306491    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adefd0a4-cd6b-4b91-9aa9-da45df2ca089-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-7b9nq\" (UID: \"adefd0a4-cd6b-4b91-9aa9-da45df2ca089\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"
Jul 15 04:17:08.308178 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.308162    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/657ae513-60d6-471a-8bb6-7d2b2e27dd32-maas-api-tls\") pod \"maas-api-664cd857c8-qv8rg\" (UID: \"657ae513-60d6-471a-8bb6-7d2b2e27dd32\") " pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:08.313632 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.313587    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n42np\" (UniqueName: \"kubernetes.io/projected/adefd0a4-cd6b-4b91-9aa9-da45df2ca089-kube-api-access-n42np\") pod \"payload-pre-processing-74584bb9f8-7b9nq\" (UID: \"adefd0a4-cd6b-4b91-9aa9-da45df2ca089\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"
Jul 15 04:17:08.406552 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.406468    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkcdv\" (UniqueName: \"kubernetes.io/projected/809162ae-0b69-4f2d-ba1a-7a97ec49f18c-kube-api-access-nkcdv\") pod \"payload-processing-f9b8d9b9b-wrt6d\" (UID: \"809162ae-0b69-4f2d-ba1a-7a97ec49f18c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"
Jul 15 04:17:08.406756 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.406561    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/809162ae-0b69-4f2d-ba1a-7a97ec49f18c-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-wrt6d\" (UID: \"809162ae-0b69-4f2d-ba1a-7a97ec49f18c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"
Jul 15 04:17:08.407291 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.407269    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/809162ae-0b69-4f2d-ba1a-7a97ec49f18c-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-wrt6d\" (UID: \"809162ae-0b69-4f2d-ba1a-7a97ec49f18c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"
Jul 15 04:17:08.414498 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.414467    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkcdv\" (UniqueName: \"kubernetes.io/projected/809162ae-0b69-4f2d-ba1a-7a97ec49f18c-kube-api-access-nkcdv\") pod \"payload-processing-f9b8d9b9b-wrt6d\" (UID: \"809162ae-0b69-4f2d-ba1a-7a97ec49f18c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"
Jul 15 04:17:08.466976 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.466940    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"
Jul 15 04:17:08.481100 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.481069    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"
Jul 15 04:17:08.541136 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.541103    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:08.620676 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.620647    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"]
Jul 15 04:17:08.625625 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:17:08.625351    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadefd0a4_cd6b_4b91_9aa9_da45df2ca089.slice/crio-2955d57faddcf71245f5b7f441e504e7d4bb340d365857519a021173fb1fe664 WatchSource:0}: Error finding container 2955d57faddcf71245f5b7f441e504e7d4bb340d365857519a021173fb1fe664: Status 404 returned error can't find the container with id 2955d57faddcf71245f5b7f441e504e7d4bb340d365857519a021173fb1fe664
Jul 15 04:17:08.653854 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.653820    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"]
Jul 15 04:17:08.704642 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:08.704615    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"]
Jul 15 04:17:08.706776 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:17:08.706749    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657ae513_60d6_471a_8bb6_7d2b2e27dd32.slice/crio-d45a3861a0ac0cd21b964a559b43ccc38d821c1c3c804c90b45a1a12759286f4 WatchSource:0}: Error finding container d45a3861a0ac0cd21b964a559b43ccc38d821c1c3c804c90b45a1a12759286f4: Status 404 returned error can't find the container with id d45a3861a0ac0cd21b964a559b43ccc38d821c1c3c804c90b45a1a12759286f4
Jul 15 04:17:09.408758 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:09.408674    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d" event={"ID":"809162ae-0b69-4f2d-ba1a-7a97ec49f18c","Type":"ContainerStarted","Data":"9bcca00041339136a136407d061ccc158d663f968bb15e0a55c811b3a21e0dc0"}
Jul 15 04:17:09.410793 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:09.410701    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq" event={"ID":"adefd0a4-cd6b-4b91-9aa9-da45df2ca089","Type":"ContainerStarted","Data":"2955d57faddcf71245f5b7f441e504e7d4bb340d365857519a021173fb1fe664"}
Jul 15 04:17:09.412435 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:09.412389    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg" event={"ID":"657ae513-60d6-471a-8bb6-7d2b2e27dd32","Type":"ContainerStarted","Data":"d45a3861a0ac0cd21b964a559b43ccc38d821c1c3c804c90b45a1a12759286f4"}
Jul 15 04:17:11.422864 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:11.422830    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d" event={"ID":"809162ae-0b69-4f2d-ba1a-7a97ec49f18c","Type":"ContainerStarted","Data":"5e1ae460b157e20f96eee512aeaba796f0c53a951eb778c466fd19c05b7cc65d"}
Jul 15 04:17:11.423253 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:11.422947    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"
Jul 15 04:17:11.424472 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:11.424446    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq" event={"ID":"adefd0a4-cd6b-4b91-9aa9-da45df2ca089","Type":"ContainerStarted","Data":"5766c82948a78f0e2636264c76f5227ad760274295ef2d26fe4948a2323799b0"}
Jul 15 04:17:11.424568 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:11.424517    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"
Jul 15 04:17:11.425877 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:11.425859    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg" event={"ID":"657ae513-60d6-471a-8bb6-7d2b2e27dd32","Type":"ContainerStarted","Data":"87dbd31c151fcccb9375f5248ea75cb13c3fc81bd6929254f03cf4f2f9a42903"}
Jul 15 04:17:11.425953 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:11.425902    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:11.438902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:11.438855    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d" podStartSLOduration=1.700371275 podStartE2EDuration="3.43884043s" totalImagesPullingTime="1.738469155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:17:11.437830134 +0000 UTC m=+660.749211382" watchObservedRunningTime="2026-07-15 04:17:11.43884043 +0000 UTC m=+660.750221677"
Jul 15 04:17:11.454324 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:11.454280    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg" podStartSLOduration=2.762756704 podStartE2EDuration="4.454266566s" totalImagesPullingTime="1.691509862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:17:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:17:11.453947646 +0000 UTC m=+660.765328894" watchObservedRunningTime="2026-07-15 04:17:11.454266566 +0000 UTC m=+660.765647813"
Jul 15 04:17:11.469973 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:11.469919    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq" podStartSLOduration=1.694927839 podStartE2EDuration="3.469901272s" totalImagesPullingTime="1.774973433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:17:11.469481833 +0000 UTC m=+660.780863082" watchObservedRunningTime="2026-07-15 04:17:11.469901272 +0000 UTC m=+660.781282520"
Jul 15 04:17:17.435867 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:17.435837    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-664cd857c8-qv8rg"
Jul 15 04:17:22.432221 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:22.432182    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-wrt6d"
Jul 15 04:17:22.432685 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:22.432268    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-7b9nq"
Jul 15 04:17:41.183363 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.183330    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"]
Jul 15 04:17:41.189052 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.189031    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.192570 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.192549    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wlffh\""
Jul 15 04:17:41.192707 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.192549    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 04:17:41.192707 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.192557    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 15 04:17:41.192707 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.192586    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 04:17:41.196447 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.196082    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"]
Jul 15 04:17:41.297761 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.297724    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.297923 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.297781    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.297923 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.297892    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.297996 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.297937    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsp7b\" (UniqueName: \"kubernetes.io/projected/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-kube-api-access-zsp7b\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.297996 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.297987    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.298061 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.298005    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.398660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.398626    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsp7b\" (UniqueName: \"kubernetes.io/projected/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-kube-api-access-zsp7b\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.398660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.398663    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.398858 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.398684    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.398858 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.398711    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.398858 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.398738    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.398858 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.398788    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.399119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.399094    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.399156 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.399106    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.399219 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.399181    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.401059 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.401035    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.401347 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.401330    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.406284 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.406257    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsp7b\" (UniqueName: \"kubernetes.io/projected/10a6c75c-71fb-4b4d-9987-b89e2c511e9a-kube-api-access-zsp7b\") pod \"e2e-trlp-test-simulated-kserve-598466548d-slhfq\" (UID: \"10a6c75c-71fb-4b4d-9987-b89e2c511e9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.500173 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.500069    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:41.635800 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.635775    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"]
Jul 15 04:17:41.637758 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:17:41.637729    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a6c75c_71fb_4b4d_9987_b89e2c511e9a.slice/crio-537e0e604d93e3854e9e9d6371fe780287aa579cc5f2b7b98f7a2a0299db5da3 WatchSource:0}: Error finding container 537e0e604d93e3854e9e9d6371fe780287aa579cc5f2b7b98f7a2a0299db5da3: Status 404 returned error can't find the container with id 537e0e604d93e3854e9e9d6371fe780287aa579cc5f2b7b98f7a2a0299db5da3
Jul 15 04:17:41.639103 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:41.639082    2524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 04:17:42.569816 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:42.569773    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq" event={"ID":"10a6c75c-71fb-4b4d-9987-b89e2c511e9a","Type":"ContainerStarted","Data":"537e0e604d93e3854e9e9d6371fe780287aa579cc5f2b7b98f7a2a0299db5da3"}
Jul 15 04:17:43.576471 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:43.576432    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq" event={"ID":"10a6c75c-71fb-4b4d-9987-b89e2c511e9a","Type":"ContainerStarted","Data":"b95dd28f4b51db7b095e3545be1d4b96f323f99438109764a9be4f39c50b08fa"}
Jul 15 04:17:43.594699 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:43.594646    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq" podStartSLOduration=1.597084528 podStartE2EDuration="2.594631516s" totalImagesPullingTime="997.546988ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:17:43.592645114 +0000 UTC m=+692.904026362" watchObservedRunningTime="2026-07-15 04:17:43.594631516 +0000 UTC m=+692.906012762"
Jul 15 04:17:51.500871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:51.500840    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:51.501337 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:51.500898    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:51.513325 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:51.513292    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:17:51.621610 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:17:51.621568    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-slhfq"
Jul 15 04:20:55.745812 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:55.745775    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67d859fc9c-g9ll2"]
Jul 15 04:20:55.746331 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:55.745981    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67d859fc9c-g9ll2" podUID="230b3b5d-8621-4c1f-8f39-52e467afdefe" containerName="manager" containerID="cri-o://8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1" gracePeriod=30
Jul 15 04:20:56.006057 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.005997    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:20:56.136283 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.136254    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/230b3b5d-8621-4c1f-8f39-52e467afdefe-webhook-cert\") pod \"230b3b5d-8621-4c1f-8f39-52e467afdefe\" (UID: \"230b3b5d-8621-4c1f-8f39-52e467afdefe\") "
Jul 15 04:20:56.136460 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.136316    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lg2x5\" (UniqueName: \"kubernetes.io/projected/230b3b5d-8621-4c1f-8f39-52e467afdefe-kube-api-access-lg2x5\") pod \"230b3b5d-8621-4c1f-8f39-52e467afdefe\" (UID: \"230b3b5d-8621-4c1f-8f39-52e467afdefe\") "
Jul 15 04:20:56.138483 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.138452    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/230b3b5d-8621-4c1f-8f39-52e467afdefe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "230b3b5d-8621-4c1f-8f39-52e467afdefe" (UID: "230b3b5d-8621-4c1f-8f39-52e467afdefe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:20:56.138623 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.138505    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/230b3b5d-8621-4c1f-8f39-52e467afdefe-kube-api-access-lg2x5" (OuterVolumeSpecName: "kube-api-access-lg2x5") pod "230b3b5d-8621-4c1f-8f39-52e467afdefe" (UID: "230b3b5d-8621-4c1f-8f39-52e467afdefe"). InnerVolumeSpecName "kube-api-access-lg2x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:20:56.237498 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.237462    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg2x5\" (UniqueName: \"kubernetes.io/projected/230b3b5d-8621-4c1f-8f39-52e467afdefe-kube-api-access-lg2x5\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:20:56.237498 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.237493    2524 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/230b3b5d-8621-4c1f-8f39-52e467afdefe-webhook-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:20:56.437740 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.437706    2524 generic.go:356] "Generic (PLEG): container finished" podID="230b3b5d-8621-4c1f-8f39-52e467afdefe" containerID="8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1" exitCode=0
Jul 15 04:20:56.437911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.437761    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67d859fc9c-g9ll2"
Jul 15 04:20:56.437911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.437796    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67d859fc9c-g9ll2" event={"ID":"230b3b5d-8621-4c1f-8f39-52e467afdefe","Type":"ContainerDied","Data":"8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1"}
Jul 15 04:20:56.437911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.437849    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67d859fc9c-g9ll2" event={"ID":"230b3b5d-8621-4c1f-8f39-52e467afdefe","Type":"ContainerDied","Data":"dee4ac5a1b7522c8c7d16f355d62eefd36420f0817c282d628c4cc3dbd7807a0"}
Jul 15 04:20:56.437911 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.437878    2524 scope.go:117] "RemoveContainer" containerID="8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1"
Jul 15 04:20:56.446526 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.446510    2524 scope.go:117] "RemoveContainer" containerID="8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1"
Jul 15 04:20:56.446802 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:20:56.446783    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1\": container with ID starting with 8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1 not found: ID does not exist" containerID="8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1"
Jul 15 04:20:56.446868 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.446810    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1"} err="failed to get container status \"8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1\": rpc error: code = NotFound desc = could not find container \"8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1\": container with ID starting with 8dd72e6d33c0661c92c2f51d1f515a3293a09f5ee95d9daddf326df8245d42d1 not found: ID does not exist"
Jul 15 04:20:56.457798 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.457772    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67d859fc9c-g9ll2"]
Jul 15 04:20:56.463557 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:56.463533    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67d859fc9c-g9ll2"]
Jul 15 04:20:57.314002 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.313965    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67d859fc9c-kh8zv"]
Jul 15 04:20:57.314385 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.314306    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="230b3b5d-8621-4c1f-8f39-52e467afdefe" containerName="manager"
Jul 15 04:20:57.314385 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.314316    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="230b3b5d-8621-4c1f-8f39-52e467afdefe" containerName="manager"
Jul 15 04:20:57.314385 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.314378    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="230b3b5d-8621-4c1f-8f39-52e467afdefe" containerName="manager"
Jul 15 04:20:57.318370 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.318350    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67d859fc9c-kh8zv"
Jul 15 04:20:57.322702 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.322673    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dwf88\""
Jul 15 04:20:57.322813 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.322711    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 04:20:57.322882 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.322853    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 04:20:57.331668 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.331647    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67d859fc9c-kh8zv"]
Jul 15 04:20:57.433807 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.433766    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="230b3b5d-8621-4c1f-8f39-52e467afdefe" path="/var/lib/kubelet/pods/230b3b5d-8621-4c1f-8f39-52e467afdefe/volumes"
Jul 15 04:20:57.448663 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.448637    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa2b64d7-833d-453c-bd68-0182c97e141d-webhook-cert\") pod \"maas-controller-67d859fc9c-kh8zv\" (UID: \"aa2b64d7-833d-453c-bd68-0182c97e141d\") " pod="opendatahub/maas-controller-67d859fc9c-kh8zv"
Jul 15 04:20:57.448812 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.448683    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4ds4\" (UniqueName: \"kubernetes.io/projected/aa2b64d7-833d-453c-bd68-0182c97e141d-kube-api-access-k4ds4\") pod \"maas-controller-67d859fc9c-kh8zv\" (UID: \"aa2b64d7-833d-453c-bd68-0182c97e141d\") " pod="opendatahub/maas-controller-67d859fc9c-kh8zv"
Jul 15 04:20:57.549992 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.549958    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa2b64d7-833d-453c-bd68-0182c97e141d-webhook-cert\") pod \"maas-controller-67d859fc9c-kh8zv\" (UID: \"aa2b64d7-833d-453c-bd68-0182c97e141d\") " pod="opendatahub/maas-controller-67d859fc9c-kh8zv"
Jul 15 04:20:57.550160 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.550009    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4ds4\" (UniqueName: \"kubernetes.io/projected/aa2b64d7-833d-453c-bd68-0182c97e141d-kube-api-access-k4ds4\") pod \"maas-controller-67d859fc9c-kh8zv\" (UID: \"aa2b64d7-833d-453c-bd68-0182c97e141d\") " pod="opendatahub/maas-controller-67d859fc9c-kh8zv"
Jul 15 04:20:57.552668 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.552644    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa2b64d7-833d-453c-bd68-0182c97e141d-webhook-cert\") pod \"maas-controller-67d859fc9c-kh8zv\" (UID: \"aa2b64d7-833d-453c-bd68-0182c97e141d\") " pod="opendatahub/maas-controller-67d859fc9c-kh8zv"
Jul 15 04:20:57.557899 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.557872    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4ds4\" (UniqueName: \"kubernetes.io/projected/aa2b64d7-833d-453c-bd68-0182c97e141d-kube-api-access-k4ds4\") pod \"maas-controller-67d859fc9c-kh8zv\" (UID: \"aa2b64d7-833d-453c-bd68-0182c97e141d\") " pod="opendatahub/maas-controller-67d859fc9c-kh8zv"
Jul 15 04:20:57.629010 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.628974    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67d859fc9c-kh8zv"
Jul 15 04:20:57.765840 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:57.765811    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67d859fc9c-kh8zv"]
Jul 15 04:20:57.768227 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:20:57.768201    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2b64d7_833d_453c_bd68_0182c97e141d.slice/crio-29dbdb02d7f761f5e9e871c868294020b43294e4f6c858c629368d7287e52f6e WatchSource:0}: Error finding container 29dbdb02d7f761f5e9e871c868294020b43294e4f6c858c629368d7287e52f6e: Status 404 returned error can't find the container with id 29dbdb02d7f761f5e9e871c868294020b43294e4f6c858c629368d7287e52f6e
Jul 15 04:20:58.447890 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:58.447846    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67d859fc9c-kh8zv" event={"ID":"aa2b64d7-833d-453c-bd68-0182c97e141d","Type":"ContainerStarted","Data":"d7e9c22711d493643e36b94b9cf980f94b6abb8cad9fa7f8d7d0ed58e9808819"}
Jul 15 04:20:58.448323 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:58.447910    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67d859fc9c-kh8zv" event={"ID":"aa2b64d7-833d-453c-bd68-0182c97e141d","Type":"ContainerStarted","Data":"29dbdb02d7f761f5e9e871c868294020b43294e4f6c858c629368d7287e52f6e"}
Jul 15 04:20:58.448323 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:58.447936    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67d859fc9c-kh8zv"
Jul 15 04:20:58.464563 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:20:58.464507    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67d859fc9c-kh8zv" podStartSLOduration=0.986069114 podStartE2EDuration="1.464490264s" totalImagesPullingTime="478.42115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:20:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:20:58.463199881 +0000 UTC m=+887.774581140" watchObservedRunningTime="2026-07-15 04:20:58.464490264 +0000 UTC m=+887.775871510"
Jul 15 04:21:09.454258 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:21:09.454228    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67d859fc9c-kh8zv"
Jul 15 04:21:11.328304 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:21:11.328276    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:21:11.328777 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:21:11.328339    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:26:11.361309 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:26:11.361276    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:26:11.361815 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:26:11.361363    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:30:00.136748 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:00.136712    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734830-g48db"]
Jul 15 04:30:00.140052 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:00.140036    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734830-g48db"
Jul 15 04:30:00.143037 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:00.143017    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jc2nz\""
Jul 15 04:30:00.149732 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:00.149714    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734830-g48db"
Jul 15 04:30:00.155091 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:00.155066    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734830-g48db"]
Jul 15 04:30:00.283482 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:00.283456    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734830-g48db"]
Jul 15 04:30:00.285463 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:30:00.285434    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce7bbf0_418f_4356_8563_51f73b2e9441.slice/crio-62af34ceb1e774c4bb843df6872cc35d552fdfb07e1630018e35eb8d84da72d2 WatchSource:0}: Error finding container 62af34ceb1e774c4bb843df6872cc35d552fdfb07e1630018e35eb8d84da72d2: Status 404 returned error can't find the container with id 62af34ceb1e774c4bb843df6872cc35d552fdfb07e1630018e35eb8d84da72d2
Jul 15 04:30:00.287410 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:00.287394    2524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 04:30:00.911581 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:00.911540    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734830-g48db" event={"ID":"2ce7bbf0-418f-4356-8563-51f73b2e9441","Type":"ContainerStarted","Data":"62af34ceb1e774c4bb843df6872cc35d552fdfb07e1630018e35eb8d84da72d2"}
Jul 15 04:30:03.929840 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:03.929741    2524 generic.go:356] "Generic (PLEG): container finished" podID="2ce7bbf0-418f-4356-8563-51f73b2e9441" containerID="2204176526f575b240a0cf0202220b9c2741d72bd05f12f9d728130546832410" exitCode=0
Jul 15 04:30:03.929840 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:03.929825    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734830-g48db" event={"ID":"2ce7bbf0-418f-4356-8563-51f73b2e9441","Type":"ContainerDied","Data":"2204176526f575b240a0cf0202220b9c2741d72bd05f12f9d728130546832410"}
Jul 15 04:30:05.067259 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:05.067233    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734830-g48db"
Jul 15 04:30:05.942324 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:05.942292    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734830-g48db"
Jul 15 04:30:05.942490 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:05.942325    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734830-g48db" event={"ID":"2ce7bbf0-418f-4356-8563-51f73b2e9441","Type":"ContainerDied","Data":"62af34ceb1e774c4bb843df6872cc35d552fdfb07e1630018e35eb8d84da72d2"}
Jul 15 04:30:05.942490 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:30:05.942371    2524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62af34ceb1e774c4bb843df6872cc35d552fdfb07e1630018e35eb8d84da72d2"
Jul 15 04:31:11.389669 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:31:11.389630    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:31:11.390311 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:31:11.390293    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:33:01.629149 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.629109    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"]
Jul 15 04:33:01.631369 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.629458    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ce7bbf0-418f-4356-8563-51f73b2e9441" containerName="cleanup"
Jul 15 04:33:01.631369 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.629469    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ce7bbf0-418f-4356-8563-51f73b2e9441" containerName="cleanup"
Jul 15 04:33:01.631369 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.629541    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ce7bbf0-418f-4356-8563-51f73b2e9441" containerName="cleanup"
Jul 15 04:33:01.632734 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.632717    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.635246 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.635223    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a3677d66-openshift-default-dockercfg-ld7r8\""
Jul 15 04:33:01.659120 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.659091    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"]
Jul 15 04:33:01.675269 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.675231    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-data\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.675438 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.675407    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-credential-socket\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.675512 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.675437    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4jxv\" (UniqueName: \"kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-kube-api-access-h4jxv\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.675512 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.675474    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-envoy\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.675633 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.675511    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-istio-token\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.675633 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.675535    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4a65a75-52a9-49c1-a078-f4604c944362-istiod-ca-cert\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.675720 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.675657    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-socket\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.675720 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.675690    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4a65a75-52a9-49c1-a078-f4604c944362-istio-podinfo\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.675720 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.675711    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-certs\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.776923 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.776889    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-socket\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777107 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.776932    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4a65a75-52a9-49c1-a078-f4604c944362-istio-podinfo\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777107 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.776961    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-certs\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777107 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.776999    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-data\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777107 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.777048    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-credential-socket\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777107 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.777073    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4jxv\" (UniqueName: \"kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-kube-api-access-h4jxv\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777107 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.777105    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-envoy\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777407 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.777138    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-istio-token\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777407 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.777160    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4a65a75-52a9-49c1-a078-f4604c944362-istiod-ca-cert\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777407 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.777335    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-socket\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777533 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.777415    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-certs\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777533 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.777448    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-data\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777641 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.777612    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-credential-socket\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.777859 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.777838    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4a65a75-52a9-49c1-a078-f4604c944362-istiod-ca-cert\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.779515 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.779491    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4a65a75-52a9-49c1-a078-f4604c944362-istio-podinfo\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.779626 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.779499    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-envoy\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.784576 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.784555    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-istio-token\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.784869 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.784852    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4jxv\" (UniqueName: \"kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-kube-api-access-h4jxv\") pod \"e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") " pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:01.944871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:01.944779    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:02.085320 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:02.085290    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"]
Jul 15 04:33:02.087129 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:33:02.087102    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4a65a75_52a9_49c1_a078_f4604c944362.slice/crio-019d65c980b858fbb65cda397d6bfc84185ceda1deb4eeaf19975fbd2b492a57 WatchSource:0}: Error finding container 019d65c980b858fbb65cda397d6bfc84185ceda1deb4eeaf19975fbd2b492a57: Status 404 returned error can't find the container with id 019d65c980b858fbb65cda397d6bfc84185ceda1deb4eeaf19975fbd2b492a57
Jul 15 04:33:02.750740 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:02.750658    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6" event={"ID":"e4a65a75-52a9-49c1-a078-f4604c944362","Type":"ContainerStarted","Data":"019d65c980b858fbb65cda397d6bfc84185ceda1deb4eeaf19975fbd2b492a57"}
Jul 15 04:33:02.944351 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:02.944312    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"]
Jul 15 04:33:02.949148 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:02.949126    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:02.953313 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:02.953283    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a3677d66-serving-cert\""
Jul 15 04:33:02.962062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:02.962041    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"]
Jul 15 04:33:02.988042 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:02.988001    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmxxc\" (UniqueName: \"kubernetes.io/projected/9a897591-0e08-4d2f-aa13-685cfb6ba819-kube-api-access-qmxxc\") pod \"maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:02.988188 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:02.988094    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a897591-0e08-4d2f-aa13-685cfb6ba819-maas-api-tls\") pod \"maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:02.988188 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:02.988120    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a897591-0e08-4d2f-aa13-685cfb6ba819-openshift-service-ca\") pod \"maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:03.089250 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:03.089154    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmxxc\" (UniqueName: \"kubernetes.io/projected/9a897591-0e08-4d2f-aa13-685cfb6ba819-kube-api-access-qmxxc\") pod \"maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:03.089250 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:03.089242    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a897591-0e08-4d2f-aa13-685cfb6ba819-maas-api-tls\") pod \"maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:03.089490 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:03.089268    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a897591-0e08-4d2f-aa13-685cfb6ba819-openshift-service-ca\") pod \"maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:03.089490 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:33:03.089394    2524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-serving-cert: secret "maas-api-e2e-ait-a3677d66-serving-cert" not found
Jul 15 04:33:03.089490 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:33:03.089461    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a897591-0e08-4d2f-aa13-685cfb6ba819-maas-api-tls podName:9a897591-0e08-4d2f-aa13-685cfb6ba819 nodeName:}" failed. No retries permitted until 2026-07-15 04:33:03.589444593 +0000 UTC m=+1612.900825821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9a897591-0e08-4d2f-aa13-685cfb6ba819-maas-api-tls") pod "maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk" (UID: "9a897591-0e08-4d2f-aa13-685cfb6ba819") : secret "maas-api-e2e-ait-a3677d66-serving-cert" not found
Jul 15 04:33:03.090067 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:03.090047    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a897591-0e08-4d2f-aa13-685cfb6ba819-openshift-service-ca\") pod \"maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:03.109254 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:03.109226    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmxxc\" (UniqueName: \"kubernetes.io/projected/9a897591-0e08-4d2f-aa13-685cfb6ba819-kube-api-access-qmxxc\") pod \"maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:03.595767 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:03.595730    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a897591-0e08-4d2f-aa13-685cfb6ba819-maas-api-tls\") pod \"maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:03.598329 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:03.598309    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a897591-0e08-4d2f-aa13-685cfb6ba819-maas-api-tls\") pod \"maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:03.861391 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:03.861308    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:04.000498 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:04.000474    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"]
Jul 15 04:33:04.002950 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:33:04.002923    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a897591_0e08_4d2f_aa13_685cfb6ba819.slice/crio-a11ef4d38d08ea8ea7d77cf24c76c352dc142dde22d518077583be1ac909dba3 WatchSource:0}: Error finding container a11ef4d38d08ea8ea7d77cf24c76c352dc142dde22d518077583be1ac909dba3: Status 404 returned error can't find the container with id a11ef4d38d08ea8ea7d77cf24c76c352dc142dde22d518077583be1ac909dba3
Jul 15 04:33:04.763004 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:04.762969    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk" event={"ID":"9a897591-0e08-4d2f-aa13-685cfb6ba819","Type":"ContainerStarted","Data":"a11ef4d38d08ea8ea7d77cf24c76c352dc142dde22d518077583be1ac909dba3"}
Jul 15 04:33:05.768681 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:05.768645    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk" event={"ID":"9a897591-0e08-4d2f-aa13-685cfb6ba819","Type":"ContainerStarted","Data":"e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99"}
Jul 15 04:33:05.769070 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:05.768731    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:05.784167 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:05.784112    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk" podStartSLOduration=2.7147124160000002 podStartE2EDuration="3.784098781s" totalImagesPullingTime="1.069386365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:33:05.783290679 +0000 UTC m=+1615.094671926" watchObservedRunningTime="2026-07-15 04:33:05.784098781 +0000 UTC m=+1615.095480028"
Jul 15 04:33:07.752704 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:07.752669    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:33:07.752962 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:07.752743    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:33:07.752962 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:07.752771    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:33:08.786864 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:08.786818    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6" event={"ID":"e4a65a75-52a9-49c1-a078-f4604c944362","Type":"ContainerStarted","Data":"39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909"}
Jul 15 04:33:08.807390 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:08.807345    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6" podStartSLOduration=2.144011433 podStartE2EDuration="7.807328364s" totalImagesPullingTime="5.663316931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:33:08.80549126 +0000 UTC m=+1618.116872508" watchObservedRunningTime="2026-07-15 04:33:08.807328364 +0000 UTC m=+1618.118709687"
Jul 15 04:33:08.945538 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:08.945501    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:08.946869 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:08.946839    2524 patch_prober.go:28] interesting pod/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 15 04:33:08.947012 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:08.946901    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6" podUID="e4a65a75-52a9-49c1-a078-f4604c944362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 15 04:33:09.945561 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:09.945522    2524 patch_prober.go:28] interesting pod/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 15 04:33:09.945957 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:09.945582    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6" podUID="e4a65a75-52a9-49c1-a078-f4604c944362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 15 04:33:10.945518 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:10.945485    2524 patch_prober.go:28] interesting pod/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 15 04:33:10.945717 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:10.945539    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6" podUID="e4a65a75-52a9-49c1-a078-f4604c944362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 15 04:33:11.783129 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:11.783097    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:11.945129 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:11.945087    2524 patch_prober.go:28] interesting pod/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 15 04:33:11.945334 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:11.945160    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6" podUID="e4a65a75-52a9-49c1-a078-f4604c944362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 15 04:33:12.948935 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:12.948903    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:12.949398 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:12.949280    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:12.949969 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:12.949949    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:18.399941 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.399901    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"]
Jul 15 04:33:18.400363 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.400105    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk" podUID="9a897591-0e08-4d2f-aa13-685cfb6ba819" containerName="maas-api" containerID="cri-o://e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99" gracePeriod=30
Jul 15 04:33:18.650936 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.650866    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:18.747954 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.747930    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmxxc\" (UniqueName: \"kubernetes.io/projected/9a897591-0e08-4d2f-aa13-685cfb6ba819-kube-api-access-qmxxc\") pod \"9a897591-0e08-4d2f-aa13-685cfb6ba819\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") "
Jul 15 04:33:18.748088 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.747993    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a897591-0e08-4d2f-aa13-685cfb6ba819-openshift-service-ca\") pod \"9a897591-0e08-4d2f-aa13-685cfb6ba819\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") "
Jul 15 04:33:18.748140 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.748108    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a897591-0e08-4d2f-aa13-685cfb6ba819-maas-api-tls\") pod \"9a897591-0e08-4d2f-aa13-685cfb6ba819\" (UID: \"9a897591-0e08-4d2f-aa13-685cfb6ba819\") "
Jul 15 04:33:18.748326 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.748303    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a897591-0e08-4d2f-aa13-685cfb6ba819-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9a897591-0e08-4d2f-aa13-685cfb6ba819" (UID: "9a897591-0e08-4d2f-aa13-685cfb6ba819"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:33:18.748468 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.748449    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a897591-0e08-4d2f-aa13-685cfb6ba819-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:18.750272 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.750250    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a897591-0e08-4d2f-aa13-685cfb6ba819-kube-api-access-qmxxc" (OuterVolumeSpecName: "kube-api-access-qmxxc") pod "9a897591-0e08-4d2f-aa13-685cfb6ba819" (UID: "9a897591-0e08-4d2f-aa13-685cfb6ba819"). InnerVolumeSpecName "kube-api-access-qmxxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:33:18.750355 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.750300    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a897591-0e08-4d2f-aa13-685cfb6ba819-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9a897591-0e08-4d2f-aa13-685cfb6ba819" (UID: "9a897591-0e08-4d2f-aa13-685cfb6ba819"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:33:18.837180 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.837140    2524 generic.go:356] "Generic (PLEG): container finished" podID="9a897591-0e08-4d2f-aa13-685cfb6ba819" containerID="e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99" exitCode=0
Jul 15 04:33:18.837306 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.837181    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"
Jul 15 04:33:18.837306 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.837233    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk" event={"ID":"9a897591-0e08-4d2f-aa13-685cfb6ba819","Type":"ContainerDied","Data":"e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99"}
Jul 15 04:33:18.837306 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.837281    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk" event={"ID":"9a897591-0e08-4d2f-aa13-685cfb6ba819","Type":"ContainerDied","Data":"a11ef4d38d08ea8ea7d77cf24c76c352dc142dde22d518077583be1ac909dba3"}
Jul 15 04:33:18.837306 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.837305    2524 scope.go:117] "RemoveContainer" containerID="e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99"
Jul 15 04:33:18.846342 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.846326    2524 scope.go:117] "RemoveContainer" containerID="e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99"
Jul 15 04:33:18.846576 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:33:18.846559    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99\": container with ID starting with e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99 not found: ID does not exist" containerID="e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99"
Jul 15 04:33:18.846635 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.846587    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99"} err="failed to get container status \"e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99\": rpc error: code = NotFound desc = could not find container \"e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99\": container with ID starting with e1316e5180fd742a74038c043fd8232718f20d08e15926b419778190975e5d99 not found: ID does not exist"
Jul 15 04:33:18.849122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.849105    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a897591-0e08-4d2f-aa13-685cfb6ba819-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:18.849190 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.849124    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmxxc\" (UniqueName: \"kubernetes.io/projected/9a897591-0e08-4d2f-aa13-685cfb6ba819-kube-api-access-qmxxc\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:18.858041 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.858020    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"]
Jul 15 04:33:18.863718 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:18.863698    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3677d66-5cbbdb9bb5-bd5kk"]
Jul 15 04:33:19.432541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:19.432511    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a897591-0e08-4d2f-aa13-685cfb6ba819" path="/var/lib/kubelet/pods/9a897591-0e08-4d2f-aa13-685cfb6ba819/volumes"
Jul 15 04:33:33.135999 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.135962    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"]
Jul 15 04:33:33.136529 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.136157    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6" podUID="e4a65a75-52a9-49c1-a078-f4604c944362" containerName="istio-proxy" containerID="cri-o://39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909" gracePeriod=30
Jul 15 04:33:33.697675 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.697634    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"]
Jul 15 04:33:33.698426 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.698399    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a897591-0e08-4d2f-aa13-685cfb6ba819" containerName="maas-api"
Jul 15 04:33:33.698426 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.698425    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a897591-0e08-4d2f-aa13-685cfb6ba819" containerName="maas-api"
Jul 15 04:33:33.698640 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.698525    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a897591-0e08-4d2f-aa13-685cfb6ba819" containerName="maas-api"
Jul 15 04:33:33.703108 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.703085    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.705405 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.705381    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-0f70507c-gw-openshift-default-dockercfg-hcz87\""
Jul 15 04:33:33.711680 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.711660    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"]
Jul 15 04:33:33.880792 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.880751    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-socket\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.880792 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.880799    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sqhg\" (UniqueName: \"kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-kube-api-access-2sqhg\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.881034 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.880858    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-credential-socket\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.881034 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.880917    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-token\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.881034 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.880935    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istiod-ca-cert\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.881034 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.880969    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-envoy\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.881034 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.880985    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-podinfo\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.881034 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.881020    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-certs\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.881266 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.881066    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-data\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.981704 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.981603    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-envoy\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.981704 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.981654    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-podinfo\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.981704 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.981697    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-certs\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.982008 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.981715    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-data\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.982008 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.981843    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-socket\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.982008 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.981895    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sqhg\" (UniqueName: \"kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-kube-api-access-2sqhg\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.982008 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.981947    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-credential-socket\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.982225 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.982011    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-token\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.982225 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.982037    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istiod-ca-cert\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.982225 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.982046    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-data\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.982225 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.982102    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-certs\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.982225 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.982123    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-socket\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.982489 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.982362    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-credential-socket\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.982746 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.982725    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istiod-ca-cert\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.984185 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.984167    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-envoy\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.984261 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.984228    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-podinfo\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.990921 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.990897    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-token\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:33.991097 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:33.991076    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sqhg\" (UniqueName: \"kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-kube-api-access-2sqhg\") pod \"e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") " pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:34.014996 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:34.014975    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:34.153176 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:34.153152    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"]
Jul 15 04:33:34.155088 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:33:34.155048    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e04ca1_c2e6_4fcd_8af0_948be5d2d4a6.slice/crio-9206b8654bc39bb1b39801f5bf23c0cb8b9d180b85d5dc8a5f2aa9d49b0dbae9 WatchSource:0}: Error finding container 9206b8654bc39bb1b39801f5bf23c0cb8b9d180b85d5dc8a5f2aa9d49b0dbae9: Status 404 returned error can't find the container with id 9206b8654bc39bb1b39801f5bf23c0cb8b9d180b85d5dc8a5f2aa9d49b0dbae9
Jul 15 04:33:34.157703 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:34.157668    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:33:34.157771 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:34.157735    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:33:34.157808 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:34.157778    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:33:34.911670 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:34.911626    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" event={"ID":"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6","Type":"ContainerStarted","Data":"d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b"}
Jul 15 04:33:34.911854 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:34.911699    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" event={"ID":"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6","Type":"ContainerStarted","Data":"9206b8654bc39bb1b39801f5bf23c0cb8b9d180b85d5dc8a5f2aa9d49b0dbae9"}
Jul 15 04:33:34.934496 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:34.934441    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" podStartSLOduration=1.9344236700000002 podStartE2EDuration="1.93442367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:33:34.933580292 +0000 UTC m=+1644.244961540" watchObservedRunningTime="2026-07-15 04:33:34.93442367 +0000 UTC m=+1644.245804917"
Jul 15 04:33:35.016156 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.016126    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:35.021852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.021828    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:35.049256 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.049228    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"]
Jul 15 04:33:35.053177 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.053157    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:35.055481 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.055456    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-0f70507c-serving-cert\""
Jul 15 04:33:35.060538 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.060513    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"]
Jul 15 04:33:35.091226 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.091194    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrczg\" (UniqueName: \"kubernetes.io/projected/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-kube-api-access-rrczg\") pod \"maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:35.091357 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.091306    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-maas-api-tls\") pod \"maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:35.091411 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.091376    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-openshift-service-ca\") pod \"maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:35.191901 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.191805    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-openshift-service-ca\") pod \"maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:35.191901 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.191869    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrczg\" (UniqueName: \"kubernetes.io/projected/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-kube-api-access-rrczg\") pod \"maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:35.192382 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.191945    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-maas-api-tls\") pod \"maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:35.192382 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:33:35.192080    2524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-serving-cert: secret "maas-api-e2e-migrate-0f70507c-serving-cert" not found
Jul 15 04:33:35.192382 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:33:35.192151    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-maas-api-tls podName:6ba9b4fc-2172-4f5d-851f-b1e02e40705a nodeName:}" failed. No retries permitted until 2026-07-15 04:33:35.692129177 +0000 UTC m=+1645.003510402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-maas-api-tls") pod "maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk" (UID: "6ba9b4fc-2172-4f5d-851f-b1e02e40705a") : secret "maas-api-e2e-migrate-0f70507c-serving-cert" not found
Jul 15 04:33:35.192582 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.192545    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-openshift-service-ca\") pod \"maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:35.206947 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.206927    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrczg\" (UniqueName: \"kubernetes.io/projected/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-kube-api-access-rrczg\") pod \"maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:35.696228 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.696192    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-maas-api-tls\") pod \"maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:35.698807 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.698774    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-maas-api-tls\") pod \"maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:35.916854 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.916803    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:35.917899 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.917877    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:35.966124 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:35.965989    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:36.107891 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:36.107847    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"]
Jul 15 04:33:36.111560 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:33:36.111531    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba9b4fc_2172_4f5d_851f_b1e02e40705a.slice/crio-dc1b2b16192444450645cfd7508f066516cb9cf4432e1b1c216405763de130bd WatchSource:0}: Error finding container dc1b2b16192444450645cfd7508f066516cb9cf4432e1b1c216405763de130bd: Status 404 returned error can't find the container with id dc1b2b16192444450645cfd7508f066516cb9cf4432e1b1c216405763de130bd
Jul 15 04:33:36.923490 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:36.923452    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk" event={"ID":"6ba9b4fc-2172-4f5d-851f-b1e02e40705a","Type":"ContainerStarted","Data":"5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194"}
Jul 15 04:33:36.923912 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:36.923507    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk" event={"ID":"6ba9b4fc-2172-4f5d-851f-b1e02e40705a","Type":"ContainerStarted","Data":"dc1b2b16192444450645cfd7508f066516cb9cf4432e1b1c216405763de130bd"}
Jul 15 04:33:36.923912 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:36.923819    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:36.944250 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:36.944187    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk" podStartSLOduration=1.500946625 podStartE2EDuration="1.944165371s" totalImagesPullingTime="443.218746ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:33:36.943554005 +0000 UTC m=+1646.254935255" watchObservedRunningTime="2026-07-15 04:33:36.944165371 +0000 UTC m=+1646.255546618"
Jul 15 04:33:38.393676 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.393646    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:38.420658 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.420384    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4jxv\" (UniqueName: \"kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-kube-api-access-h4jxv\") pod \"e4a65a75-52a9-49c1-a078-f4604c944362\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") "
Jul 15 04:33:38.420658 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.420469    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-istio-token\") pod \"e4a65a75-52a9-49c1-a078-f4604c944362\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") "
Jul 15 04:33:38.420658 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.420496    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-socket\") pod \"e4a65a75-52a9-49c1-a078-f4604c944362\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") "
Jul 15 04:33:38.420658 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.420537    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-credential-socket\") pod \"e4a65a75-52a9-49c1-a078-f4604c944362\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") "
Jul 15 04:33:38.420658 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.420627    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-data\") pod \"e4a65a75-52a9-49c1-a078-f4604c944362\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") "
Jul 15 04:33:38.420658 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.420656    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4a65a75-52a9-49c1-a078-f4604c944362-istiod-ca-cert\") pod \"e4a65a75-52a9-49c1-a078-f4604c944362\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") "
Jul 15 04:33:38.421065 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.420699    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-certs\") pod \"e4a65a75-52a9-49c1-a078-f4604c944362\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") "
Jul 15 04:33:38.421065 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.420743    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4a65a75-52a9-49c1-a078-f4604c944362-istio-podinfo\") pod \"e4a65a75-52a9-49c1-a078-f4604c944362\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") "
Jul 15 04:33:38.421065 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.420773    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-envoy\") pod \"e4a65a75-52a9-49c1-a078-f4604c944362\" (UID: \"e4a65a75-52a9-49c1-a078-f4604c944362\") "
Jul 15 04:33:38.421842 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.421471    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4a65a75-52a9-49c1-a078-f4604c944362" (UID: "e4a65a75-52a9-49c1-a078-f4604c944362"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:33:38.421842 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.421487    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4a65a75-52a9-49c1-a078-f4604c944362" (UID: "e4a65a75-52a9-49c1-a078-f4604c944362"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:33:38.421842 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.421772    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4a65a75-52a9-49c1-a078-f4604c944362-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4a65a75-52a9-49c1-a078-f4604c944362" (UID: "e4a65a75-52a9-49c1-a078-f4604c944362"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:33:38.422080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.422048    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4a65a75-52a9-49c1-a078-f4604c944362" (UID: "e4a65a75-52a9-49c1-a078-f4604c944362"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:33:38.422217 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.422192    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4a65a75-52a9-49c1-a078-f4604c944362" (UID: "e4a65a75-52a9-49c1-a078-f4604c944362"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:33:38.425259 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.425029    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4a65a75-52a9-49c1-a078-f4604c944362" (UID: "e4a65a75-52a9-49c1-a078-f4604c944362"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:33:38.425259 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.425165    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:38.425259 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.425184    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4a65a75-52a9-49c1-a078-f4604c944362-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:38.425259 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.425198    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:38.425259 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.425212    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:38.425259 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.425224    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:38.425259 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.425237    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:38.426466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.426442    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4a65a75-52a9-49c1-a078-f4604c944362-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4a65a75-52a9-49c1-a078-f4604c944362" (UID: "e4a65a75-52a9-49c1-a078-f4604c944362"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:33:38.426559 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.426464    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-kube-api-access-h4jxv" (OuterVolumeSpecName: "kube-api-access-h4jxv") pod "e4a65a75-52a9-49c1-a078-f4604c944362" (UID: "e4a65a75-52a9-49c1-a078-f4604c944362"). InnerVolumeSpecName "kube-api-access-h4jxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:33:38.426784 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.426754    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4a65a75-52a9-49c1-a078-f4604c944362" (UID: "e4a65a75-52a9-49c1-a078-f4604c944362"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:33:38.526231 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.526202    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4a65a75-52a9-49c1-a078-f4604c944362-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:38.526231 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.526229    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4a65a75-52a9-49c1-a078-f4604c944362-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:38.526231 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.526238    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4jxv\" (UniqueName: \"kubernetes.io/projected/e4a65a75-52a9-49c1-a078-f4604c944362-kube-api-access-h4jxv\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:33:38.940167 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.940129    2524 generic.go:356] "Generic (PLEG): container finished" podID="e4a65a75-52a9-49c1-a078-f4604c944362" containerID="39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909" exitCode=0
Jul 15 04:33:38.940343 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.940179    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"
Jul 15 04:33:38.940343 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.940204    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6" event={"ID":"e4a65a75-52a9-49c1-a078-f4604c944362","Type":"ContainerDied","Data":"39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909"}
Jul 15 04:33:38.940343 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.940282    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6" event={"ID":"e4a65a75-52a9-49c1-a078-f4604c944362","Type":"ContainerDied","Data":"019d65c980b858fbb65cda397d6bfc84185ceda1deb4eeaf19975fbd2b492a57"}
Jul 15 04:33:38.940343 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.940304    2524 scope.go:117] "RemoveContainer" containerID="39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909"
Jul 15 04:33:38.950003 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.949988    2524 scope.go:117] "RemoveContainer" containerID="39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909"
Jul 15 04:33:38.950233 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:33:38.950218    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909\": container with ID starting with 39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909 not found: ID does not exist" containerID="39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909"
Jul 15 04:33:38.950294 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.950240    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909"} err="failed to get container status \"39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909\": rpc error: code = NotFound desc = could not find container \"39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909\": container with ID starting with 39aac5146499f4ba73bc35be4ba8ddb70aeb9197a554e83a5eaa4f1b6739b909 not found: ID does not exist"
Jul 15 04:33:38.965099 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.962389    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"]
Jul 15 04:33:38.967588 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:38.967566    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a3677d66-openshift-default-67b6fd46c-8ppd6"]
Jul 15 04:33:39.432436 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:39.432406    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4a65a75-52a9-49c1-a078-f4604c944362" path="/var/lib/kubelet/pods/e4a65a75-52a9-49c1-a078-f4604c944362/volumes"
Jul 15 04:33:42.934391 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:42.934363    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:33:51.163694 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:51.163002    2524 patch_prober.go:28] interesting pod/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": read tcp 10.133.0.2:51534->10.133.0.42:15021: read: connection reset by peer" start-of-body=
Jul 15 04:33:51.163694 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:51.163067    2524 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": read tcp 10.133.0.2:51534->10.133.0.42:15021: read: connection reset by peer"
Jul 15 04:33:52.010182 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:52.010144    2524 generic.go:356] "Generic (PLEG): container finished" podID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerID="d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b" exitCode=0
Jul 15 04:33:52.010367 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:52.010219    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" event={"ID":"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6","Type":"ContainerDied","Data":"d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b"}
Jul 15 04:33:52.010585 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:52.010570    2524 scope.go:117] "RemoveContainer" containerID="d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b"
Jul 15 04:33:52.011613 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:52.011567    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:33:52.011691 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:52.011671    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:33:52.011746 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:52.011729    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:33:53.016988 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:53.016952    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" event={"ID":"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6","Type":"ContainerStarted","Data":"5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2"}
Jul 15 04:33:54.015683 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:54.015641    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:54.015944 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:54.015920    2524 patch_prober.go:28] interesting pod/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 15 04:33:54.016032 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:54.015978    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 15 04:33:55.016057 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:55.016019    2524 patch_prober.go:28] interesting pod/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 15 04:33:55.016440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:55.016084    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 15 04:33:56.015987 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:56.015954    2524 patch_prober.go:28] interesting pod/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 15 04:33:56.016169 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:56.016009    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 15 04:33:57.016308 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:57.016267    2524 patch_prober.go:28] interesting pod/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 15 04:33:57.016731 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:57.016322    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 15 04:33:58.019124 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:58.019092    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:58.019501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:58.019368    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:33:58.020105 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:33:58.020088    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:34:39.596089 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:39.596051    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"]
Jul 15 04:34:39.596578 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:39.596308    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy" containerID="cri-o://5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2" gracePeriod=30
Jul 15 04:34:43.022032 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:43.021987    2524 patch_prober.go:28] interesting pod/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 04:34:43.022444 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:43.022072    2524 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 04:34:44.753432 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.753384    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"]
Jul 15 04:34:44.754218 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.754184    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk" podUID="6ba9b4fc-2172-4f5d-851f-b1e02e40705a" containerName="maas-api" containerID="cri-o://5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194" gracePeriod=30
Jul 15 04:34:44.879408 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.879381    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:34:44.986247 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.986219    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-data\") pod \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") "
Jul 15 04:34:44.986437 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.986263    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-certs\") pod \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") "
Jul 15 04:34:44.986437 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.986292    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istiod-ca-cert\") pod \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") "
Jul 15 04:34:44.986437 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.986312    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-socket\") pod \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") "
Jul 15 04:34:44.986437 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.986351    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-envoy\") pod \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") "
Jul 15 04:34:44.986437 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.986391    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-credential-socket\") pod \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") "
Jul 15 04:34:44.986737 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.986470    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-token\") pod \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") "
Jul 15 04:34:44.986737 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.986493    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-podinfo\") pod \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") "
Jul 15 04:34:44.986737 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.986527    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sqhg\" (UniqueName: \"kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-kube-api-access-2sqhg\") pod \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\" (UID: \"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6\") "
Jul 15 04:34:44.986737 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.986691    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" (UID: "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:34:44.986934 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.986903    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:44.987061 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.987014    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" (UID: "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:34:44.987307 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.987216    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" (UID: "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:34:44.987307 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.987245    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" (UID: "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:34:44.987614 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.987404    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" (UID: "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:34:44.989148 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.989119    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" (UID: "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:34:44.989389 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.989367    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-kube-api-access-2sqhg" (OuterVolumeSpecName: "kube-api-access-2sqhg") pod "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" (UID: "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6"). InnerVolumeSpecName "kube-api-access-2sqhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:34:44.989461 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.989403    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" (UID: "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:34:44.989635 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:44.989585    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" (UID: "b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:34:45.005303 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.005283    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:34:45.087997 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.087972    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-openshift-service-ca\") pod \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") "
Jul 15 04:34:45.088155 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.088029    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-maas-api-tls\") pod \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") "
Jul 15 04:34:45.088155 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.088059    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrczg\" (UniqueName: \"kubernetes.io/projected/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-kube-api-access-rrczg\") pod \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\" (UID: \"6ba9b4fc-2172-4f5d-851f-b1e02e40705a\") "
Jul 15 04:34:45.088279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.088220    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:45.088279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.088231    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:45.088279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.088240    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sqhg\" (UniqueName: \"kubernetes.io/projected/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-kube-api-access-2sqhg\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:45.088279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.088247    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:45.088279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.088255    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:45.088279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.088263    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:45.088279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.088271    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:45.088279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.088279    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:45.088581 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.088279    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6ba9b4fc-2172-4f5d-851f-b1e02e40705a" (UID: "6ba9b4fc-2172-4f5d-851f-b1e02e40705a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:34:45.090239 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.090216    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-kube-api-access-rrczg" (OuterVolumeSpecName: "kube-api-access-rrczg") pod "6ba9b4fc-2172-4f5d-851f-b1e02e40705a" (UID: "6ba9b4fc-2172-4f5d-851f-b1e02e40705a"). InnerVolumeSpecName "kube-api-access-rrczg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:34:45.090340 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.090271    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6ba9b4fc-2172-4f5d-851f-b1e02e40705a" (UID: "6ba9b4fc-2172-4f5d-851f-b1e02e40705a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:34:45.189569 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.189544    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:45.189705 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.189570    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:45.189705 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.189583    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrczg\" (UniqueName: \"kubernetes.io/projected/6ba9b4fc-2172-4f5d-851f-b1e02e40705a-kube-api-access-rrczg\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:34:45.263738 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.263698    2524 generic.go:356] "Generic (PLEG): container finished" podID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerID="5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2" exitCode=0
Jul 15 04:34:45.263889 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.263764    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"
Jul 15 04:34:45.263889 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.263784    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" event={"ID":"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6","Type":"ContainerDied","Data":"5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2"}
Jul 15 04:34:45.263980 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.263885    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct" event={"ID":"b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6","Type":"ContainerDied","Data":"9206b8654bc39bb1b39801f5bf23c0cb8b9d180b85d5dc8a5f2aa9d49b0dbae9"}
Jul 15 04:34:45.263980 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.263920    2524 scope.go:117] "RemoveContainer" containerID="5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2"
Jul 15 04:34:45.265105 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.265081    2524 generic.go:356] "Generic (PLEG): container finished" podID="6ba9b4fc-2172-4f5d-851f-b1e02e40705a" containerID="5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194" exitCode=0
Jul 15 04:34:45.265210 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.265141    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk" event={"ID":"6ba9b4fc-2172-4f5d-851f-b1e02e40705a","Type":"ContainerDied","Data":"5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194"}
Jul 15 04:34:45.265210 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.265146    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"
Jul 15 04:34:45.265210 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.265178    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk" event={"ID":"6ba9b4fc-2172-4f5d-851f-b1e02e40705a","Type":"ContainerDied","Data":"dc1b2b16192444450645cfd7508f066516cb9cf4432e1b1c216405763de130bd"}
Jul 15 04:34:45.273015 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.272995    2524 scope.go:117] "RemoveContainer" containerID="d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b"
Jul 15 04:34:45.281265 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.281250    2524 scope.go:117] "RemoveContainer" containerID="5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2"
Jul 15 04:34:45.281486 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:34:45.281468    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2\": container with ID starting with 5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2 not found: ID does not exist" containerID="5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2"
Jul 15 04:34:45.281558 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.281496    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2"} err="failed to get container status \"5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2\": rpc error: code = NotFound desc = could not find container \"5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2\": container with ID starting with 5c3a9207cef86c8bd6a92b83b4cc7bf0f22cd666562b7d5fb7ee4f335e08a4a2 not found: ID does not exist"
Jul 15 04:34:45.281558 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.281518    2524 scope.go:117] "RemoveContainer" containerID="d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b"
Jul 15 04:34:45.281790 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:34:45.281771    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b\": container with ID starting with d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b not found: ID does not exist" containerID="d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b"
Jul 15 04:34:45.281843 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.281802    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b"} err="failed to get container status \"d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b\": rpc error: code = NotFound desc = could not find container \"d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b\": container with ID starting with d9a677db5aed6735daa60f450350f9cc3938bc7eab56b80164e84d0af07a478b not found: ID does not exist"
Jul 15 04:34:45.281843 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.281823    2524 scope.go:117] "RemoveContainer" containerID="5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194"
Jul 15 04:34:45.288336 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.288314    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"]
Jul 15 04:34:45.290067 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.290038    2524 scope.go:117] "RemoveContainer" containerID="5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194"
Jul 15 04:34:45.290767 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:34:45.290647    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194\": container with ID starting with 5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194 not found: ID does not exist" containerID="5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194"
Jul 15 04:34:45.290970 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.290710    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194"} err="failed to get container status \"5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194\": rpc error: code = NotFound desc = could not find container \"5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194\": container with ID starting with 5985b017b213cf5a323262fa60b20aa68ca063e778b0cb788b8149e91775d194 not found: ID does not exist"
Jul 15 04:34:45.292700 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.292679    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-0f70507c-gw-openshift-default-6c9ff7c7cf-gznct"]
Jul 15 04:34:45.302302 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.302283    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"]
Jul 15 04:34:45.307966 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.307946    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f70507c-7469759d8b-gkckk"]
Jul 15 04:34:45.431644 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.431615    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ba9b4fc-2172-4f5d-851f-b1e02e40705a" path="/var/lib/kubelet/pods/6ba9b4fc-2172-4f5d-851f-b1e02e40705a/volumes"
Jul 15 04:34:45.432031 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:45.432018    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" path="/var/lib/kubelet/pods/b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6/volumes"
Jul 15 04:34:51.362679 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.362644    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"]
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363468    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ba9b4fc-2172-4f5d-851f-b1e02e40705a" containerName="maas-api"
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363501    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ba9b4fc-2172-4f5d-851f-b1e02e40705a" containerName="maas-api"
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363547    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy"
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363557    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy"
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363581    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4a65a75-52a9-49c1-a078-f4604c944362" containerName="istio-proxy"
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363590    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4a65a75-52a9-49c1-a078-f4604c944362" containerName="istio-proxy"
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363620    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy"
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363636    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy"
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363820    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ba9b4fc-2172-4f5d-851f-b1e02e40705a" containerName="maas-api"
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363837    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy"
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363856    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6e04ca1-c2e6-4fcd-8af0-948be5d2d4a6" containerName="istio-proxy"
Jul 15 04:34:51.363965 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.363867    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4a65a75-52a9-49c1-a078-f4604c944362" containerName="istio-proxy"
Jul 15 04:34:51.369640 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.369579    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.372376 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.372352    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c27a526e-openshift-default-dockercfg-bc5pf\""
Jul 15 04:34:51.375800 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.375775    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"]
Jul 15 04:34:51.438940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.438915    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-socket\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.439068 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.438943    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-credential-socket\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.439068 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.438963    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-podinfo\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.439068 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.439003    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-data\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.439068 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.439041    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c9x2\" (UniqueName: \"kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-kube-api-access-8c9x2\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.439208 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.439074    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-certs\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.439208 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.439091    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-token\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.439208 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.439109    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-envoy\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.439208 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.439138    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istiod-ca-cert\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540000 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.539972    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istiod-ca-cert\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540139 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540015    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-socket\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540139 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540030    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-credential-socket\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540139 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540048    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-podinfo\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540139 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540097    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-data\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540336 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540137    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c9x2\" (UniqueName: \"kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-kube-api-access-8c9x2\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540336 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540174    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-certs\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540336 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540194    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-token\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540336 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540217    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-envoy\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540524 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540420    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-socket\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540587 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540554    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-credential-socket\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540692 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540633    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-certs\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.540692 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.540661    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-data\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.541118 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.541094    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istiod-ca-cert\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.542563 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.542546    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-envoy\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.542816 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.542794    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-podinfo\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.548310 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.548283    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c9x2\" (UniqueName: \"kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-kube-api-access-8c9x2\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.548465 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.548449    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-token\") pod \"e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") " pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.682412 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.682379    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:51.817340 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.817315    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"]
Jul 15 04:34:51.819319 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:34:51.819293    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc89b67d_1a23_4597_ad33_f7b9c15423f6.slice/crio-7a7d5947b95d670a1c468f6c3b0ba3c4233c361325c108d3645aa7d5e3203c6a WatchSource:0}: Error finding container 7a7d5947b95d670a1c468f6c3b0ba3c4233c361325c108d3645aa7d5e3203c6a: Status 404 returned error can't find the container with id 7a7d5947b95d670a1c468f6c3b0ba3c4233c361325c108d3645aa7d5e3203c6a
Jul 15 04:34:51.821865 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.821828    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:34:51.821954 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.821896    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:34:51.821954 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:51.821946    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:34:52.300613 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.300553    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" event={"ID":"bc89b67d-1a23-4597-ad33-f7b9c15423f6","Type":"ContainerStarted","Data":"fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95"}
Jul 15 04:34:52.300780 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.300624    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" event={"ID":"bc89b67d-1a23-4597-ad33-f7b9c15423f6","Type":"ContainerStarted","Data":"7a7d5947b95d670a1c468f6c3b0ba3c4233c361325c108d3645aa7d5e3203c6a"}
Jul 15 04:34:52.322320 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.322276    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" podStartSLOduration=1.322260129 podStartE2EDuration="1.322260129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:34:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:34:52.320148232 +0000 UTC m=+1721.631529481" watchObservedRunningTime="2026-07-15 04:34:52.322260129 +0000 UTC m=+1721.633641439"
Jul 15 04:34:52.682849 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.682817    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:52.687445 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.687419    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:52.701861 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.701835    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"]
Jul 15 04:34:52.705355 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.705337    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:52.707647 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.707626    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c27a526e-serving-cert\""
Jul 15 04:34:52.714525 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.714497    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"]
Jul 15 04:34:52.749958 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.749927    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfdcc\" (UniqueName: \"kubernetes.io/projected/70695052-2945-409e-be7b-82025fd101cd-kube-api-access-rfdcc\") pod \"maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:52.750094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.749968    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70695052-2945-409e-be7b-82025fd101cd-maas-api-tls\") pod \"maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:52.750094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.750052    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70695052-2945-409e-be7b-82025fd101cd-openshift-service-ca\") pod \"maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:52.850792 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.850757    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70695052-2945-409e-be7b-82025fd101cd-openshift-service-ca\") pod \"maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:52.850958 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.850827    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfdcc\" (UniqueName: \"kubernetes.io/projected/70695052-2945-409e-be7b-82025fd101cd-kube-api-access-rfdcc\") pod \"maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:52.850958 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.850856    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70695052-2945-409e-be7b-82025fd101cd-maas-api-tls\") pod \"maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:52.851094 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:34:52.850973    2524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-serving-cert: secret "maas-api-e2e-ait-c27a526e-serving-cert" not found
Jul 15 04:34:52.851094 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:34:52.851045    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70695052-2945-409e-be7b-82025fd101cd-maas-api-tls podName:70695052-2945-409e-be7b-82025fd101cd nodeName:}" failed. No retries permitted until 2026-07-15 04:34:53.351023891 +0000 UTC m=+1722.662405118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70695052-2945-409e-be7b-82025fd101cd-maas-api-tls") pod "maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd" (UID: "70695052-2945-409e-be7b-82025fd101cd") : secret "maas-api-e2e-ait-c27a526e-serving-cert" not found
Jul 15 04:34:52.851417 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.851397    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70695052-2945-409e-be7b-82025fd101cd-openshift-service-ca\") pod \"maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:52.859466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:52.859442    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfdcc\" (UniqueName: \"kubernetes.io/projected/70695052-2945-409e-be7b-82025fd101cd-kube-api-access-rfdcc\") pod \"maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:53.306825 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:53.306784    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:53.307784 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:53.307762    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:34:53.355034 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:53.354997    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70695052-2945-409e-be7b-82025fd101cd-maas-api-tls\") pod \"maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:53.358012 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:53.357983    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70695052-2945-409e-be7b-82025fd101cd-maas-api-tls\") pod \"maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:53.619955 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:53.619924    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:53.755872 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:53.755848    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"]
Jul 15 04:34:53.757785 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:34:53.757754    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70695052_2945_409e_be7b_82025fd101cd.slice/crio-dc7cb4e7e30b78c2e54c65d3b5b8eeeba1419e243827c82383cf693818c1f0f6 WatchSource:0}: Error finding container dc7cb4e7e30b78c2e54c65d3b5b8eeeba1419e243827c82383cf693818c1f0f6: Status 404 returned error can't find the container with id dc7cb4e7e30b78c2e54c65d3b5b8eeeba1419e243827c82383cf693818c1f0f6
Jul 15 04:34:54.312549 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:54.312508    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd" event={"ID":"70695052-2945-409e-be7b-82025fd101cd","Type":"ContainerStarted","Data":"96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10"}
Jul 15 04:34:54.312734 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:54.312579    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd" event={"ID":"70695052-2945-409e-be7b-82025fd101cd","Type":"ContainerStarted","Data":"dc7cb4e7e30b78c2e54c65d3b5b8eeeba1419e243827c82383cf693818c1f0f6"}
Jul 15 04:34:54.312802 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:54.312795    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:34:54.330673 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:34:54.330621    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd" podStartSLOduration=1.973175791 podStartE2EDuration="2.33058972s" totalImagesPullingTime="357.413929ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:34:54.328902857 +0000 UTC m=+1723.640284104" watchObservedRunningTime="2026-07-15 04:34:54.33058972 +0000 UTC m=+1723.641970968"
Jul 15 04:35:00.325205 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:00.325175    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:35:04.365410 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:04.365319    2524 generic.go:356] "Generic (PLEG): container finished" podID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerID="fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95" exitCode=0
Jul 15 04:35:04.365410 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:04.365390    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" event={"ID":"bc89b67d-1a23-4597-ad33-f7b9c15423f6","Type":"ContainerDied","Data":"fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95"}
Jul 15 04:35:04.365849 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:04.365816    2524 scope.go:117] "RemoveContainer" containerID="fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95"
Jul 15 04:35:04.366907 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:04.366836    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:35:04.366907 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:04.366903    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:35:04.367014 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:04.366934    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:35:05.377480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:05.377432    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" event={"ID":"bc89b67d-1a23-4597-ad33-f7b9c15423f6","Type":"ContainerStarted","Data":"9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec"}
Jul 15 04:35:05.683366 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:05.683268    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:35:05.684568 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:05.684545    2524 patch_prober.go:28] interesting pod/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 15 04:35:05.684658 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:05.684636    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 15 04:35:06.683777 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:06.683734    2524 patch_prober.go:28] interesting pod/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 15 04:35:06.684159 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:06.683820    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 15 04:35:07.682967 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:07.682927    2524 patch_prober.go:28] interesting pod/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 15 04:35:07.683153 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:07.682985    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 15 04:35:08.307141 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:08.307100    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:35:08.683506 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:08.683462    2524 patch_prober.go:28] interesting pod/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 15 04:35:08.683714 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:08.683527    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 15 04:35:09.686657 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:09.686627    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:35:09.687863 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:35:09.687838    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:36:08.089126 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.089034    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"]
Jul 15 04:36:08.089726 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.089306    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd" podUID="70695052-2945-409e-be7b-82025fd101cd" containerName="maas-api" containerID="cri-o://96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10" gracePeriod=30
Jul 15 04:36:08.347716 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.347661    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:36:08.511813 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.511766    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfdcc\" (UniqueName: \"kubernetes.io/projected/70695052-2945-409e-be7b-82025fd101cd-kube-api-access-rfdcc\") pod \"70695052-2945-409e-be7b-82025fd101cd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") "
Jul 15 04:36:08.511813 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.511818    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70695052-2945-409e-be7b-82025fd101cd-maas-api-tls\") pod \"70695052-2945-409e-be7b-82025fd101cd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") "
Jul 15 04:36:08.512076 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.511868    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70695052-2945-409e-be7b-82025fd101cd-openshift-service-ca\") pod \"70695052-2945-409e-be7b-82025fd101cd\" (UID: \"70695052-2945-409e-be7b-82025fd101cd\") "
Jul 15 04:36:08.512282 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.512256    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70695052-2945-409e-be7b-82025fd101cd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70695052-2945-409e-be7b-82025fd101cd" (UID: "70695052-2945-409e-be7b-82025fd101cd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:36:08.514009 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.513982    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70695052-2945-409e-be7b-82025fd101cd-kube-api-access-rfdcc" (OuterVolumeSpecName: "kube-api-access-rfdcc") pod "70695052-2945-409e-be7b-82025fd101cd" (UID: "70695052-2945-409e-be7b-82025fd101cd"). InnerVolumeSpecName "kube-api-access-rfdcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:36:08.514009 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.513996    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70695052-2945-409e-be7b-82025fd101cd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70695052-2945-409e-be7b-82025fd101cd" (UID: "70695052-2945-409e-be7b-82025fd101cd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:36:08.612769 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.612689    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfdcc\" (UniqueName: \"kubernetes.io/projected/70695052-2945-409e-be7b-82025fd101cd-kube-api-access-rfdcc\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:08.612769 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.612718    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70695052-2945-409e-be7b-82025fd101cd-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:08.612769 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.612731    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70695052-2945-409e-be7b-82025fd101cd-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:08.667857 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.667827    2524 generic.go:356] "Generic (PLEG): container finished" podID="70695052-2945-409e-be7b-82025fd101cd" containerID="96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10" exitCode=0
Jul 15 04:36:08.667989 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.667870    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"
Jul 15 04:36:08.667989 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.667912    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd" event={"ID":"70695052-2945-409e-be7b-82025fd101cd","Type":"ContainerDied","Data":"96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10"}
Jul 15 04:36:08.667989 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.667964    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd" event={"ID":"70695052-2945-409e-be7b-82025fd101cd","Type":"ContainerDied","Data":"dc7cb4e7e30b78c2e54c65d3b5b8eeeba1419e243827c82383cf693818c1f0f6"}
Jul 15 04:36:08.668098 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.667990    2524 scope.go:117] "RemoveContainer" containerID="96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10"
Jul 15 04:36:08.677113 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.677098    2524 scope.go:117] "RemoveContainer" containerID="96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10"
Jul 15 04:36:08.677389 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:36:08.677371    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10\": container with ID starting with 96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10 not found: ID does not exist" containerID="96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10"
Jul 15 04:36:08.677456 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.677401    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10"} err="failed to get container status \"96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10\": rpc error: code = NotFound desc = could not find container \"96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10\": container with ID starting with 96032dd64cb2fee8ea61eccf94a297d4c09643d4fa4305e8288a0655ba17ba10 not found: ID does not exist"
Jul 15 04:36:08.688979 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.688959    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"]
Jul 15 04:36:08.696617 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:08.695019    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c27a526e-5c95b5794b-8l4nd"]
Jul 15 04:36:09.432400 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:09.432370    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70695052-2945-409e-be7b-82025fd101cd" path="/var/lib/kubelet/pods/70695052-2945-409e-be7b-82025fd101cd/volumes"
Jul 15 04:36:11.417956 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:11.417924    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:36:11.419945 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:11.419917    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:36:23.801139 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:23.801109    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"]
Jul 15 04:36:23.801554 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:23.801357    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy" containerID="cri-o://9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec" gracePeriod=30
Jul 15 04:36:24.106771 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.106688    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"]
Jul 15 04:36:24.107056 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.107045    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70695052-2945-409e-be7b-82025fd101cd" containerName="maas-api"
Jul 15 04:36:24.107118 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.107058    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="70695052-2945-409e-be7b-82025fd101cd" containerName="maas-api"
Jul 15 04:36:24.107157 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.107129    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="70695052-2945-409e-be7b-82025fd101cd" containerName="maas-api"
Jul 15 04:36:24.110085 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.110063    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.112404 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.112380    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-f6197270-openshift-default-dockercfg-n6qxp\""
Jul 15 04:36:24.122357 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.122331    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"]
Jul 15 04:36:24.143574 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.143544    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-istio-token\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.143726 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.143640    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-credential-socket\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.143726 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.143666    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-envoy\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.143726 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.143721    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7913969e-b095-47dd-8fa5-48d151806141-istio-podinfo\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.143907 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.143759    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-socket\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.143907 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.143788    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-certs\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.143907 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.143809    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-data\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.143907 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.143839    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7913969e-b095-47dd-8fa5-48d151806141-istiod-ca-cert\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.143907 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.143865    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9kww\" (UniqueName: \"kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-kube-api-access-z9kww\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.244675 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.244641    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-istio-token\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.244852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.244692    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-credential-socket\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.244852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.244714    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-envoy\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.244852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.244763    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7913969e-b095-47dd-8fa5-48d151806141-istio-podinfo\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.244852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.244798    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-socket\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.244852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.244824    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-certs\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.245121 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.244948    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-data\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.245121 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.245000    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7913969e-b095-47dd-8fa5-48d151806141-istiod-ca-cert\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.245121 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.245031    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9kww\" (UniqueName: \"kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-kube-api-access-z9kww\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.245121 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.245100    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-credential-socket\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.245318 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.245209    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-certs\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.245318 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.245294    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-socket\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.245458 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.245441    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-data\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.245972 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.245951    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7913969e-b095-47dd-8fa5-48d151806141-istiod-ca-cert\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.247263 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.247235    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-envoy\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.247384 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.247366    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7913969e-b095-47dd-8fa5-48d151806141-istio-podinfo\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.252945 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.252914    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9kww\" (UniqueName: \"kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-kube-api-access-z9kww\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.253044 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.253028    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-istio-token\") pod \"e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") " pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.422078 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.422045    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:24.558486 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.558438    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"]
Jul 15 04:36:24.563652 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:36:24.562273    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7913969e_b095_47dd_8fa5_48d151806141.slice/crio-22c10f7c44ff19f81a5da1e9904075f39841cc0e3b61e2c74e5af2129e3cdc01 WatchSource:0}: Error finding container 22c10f7c44ff19f81a5da1e9904075f39841cc0e3b61e2c74e5af2129e3cdc01: Status 404 returned error can't find the container with id 22c10f7c44ff19f81a5da1e9904075f39841cc0e3b61e2c74e5af2129e3cdc01
Jul 15 04:36:24.565122 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.565086    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:36:24.565222 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.565155    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:36:24.565222 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.565188    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:36:24.689829 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.689740    2524 patch_prober.go:28] interesting pod/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 04:36:24.689829 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.689813    2524 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 04:36:24.740110 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.740070    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" event={"ID":"7913969e-b095-47dd-8fa5-48d151806141","Type":"ContainerStarted","Data":"acea53cc00afade44b9ccf88906eec0ffa9d4655a68a102a2f658d5b9a70a424"}
Jul 15 04:36:24.740279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.740137    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" event={"ID":"7913969e-b095-47dd-8fa5-48d151806141","Type":"ContainerStarted","Data":"22c10f7c44ff19f81a5da1e9904075f39841cc0e3b61e2c74e5af2129e3cdc01"}
Jul 15 04:36:24.759119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:24.759073    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" podStartSLOduration=0.759059172 podStartE2EDuration="759.059172ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:36:24.757167844 +0000 UTC m=+1814.068549093" watchObservedRunningTime="2026-07-15 04:36:24.759059172 +0000 UTC m=+1814.070440420"
Jul 15 04:36:25.326358 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.326328    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"]
Jul 15 04:36:25.330218 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.330198    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:25.333154 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.333130    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-f6197270-serving-cert\""
Jul 15 04:36:25.340787 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.340764    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"]
Jul 15 04:36:25.352975 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.352950    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/204a06bb-dc05-44d9-b667-69cbbf537982-openshift-service-ca\") pod \"maas-api-e2e-derive-f6197270-79997b9d58-5qbc2\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:25.353097 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.352985    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/204a06bb-dc05-44d9-b667-69cbbf537982-maas-api-tls\") pod \"maas-api-e2e-derive-f6197270-79997b9d58-5qbc2\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:25.353162 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.353095    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8wtt\" (UniqueName: \"kubernetes.io/projected/204a06bb-dc05-44d9-b667-69cbbf537982-kube-api-access-q8wtt\") pod \"maas-api-e2e-derive-f6197270-79997b9d58-5qbc2\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:25.423217 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.423187    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:25.432997 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.432977    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:25.453662 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.453624    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8wtt\" (UniqueName: \"kubernetes.io/projected/204a06bb-dc05-44d9-b667-69cbbf537982-kube-api-access-q8wtt\") pod \"maas-api-e2e-derive-f6197270-79997b9d58-5qbc2\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:25.453763 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.453707    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/204a06bb-dc05-44d9-b667-69cbbf537982-openshift-service-ca\") pod \"maas-api-e2e-derive-f6197270-79997b9d58-5qbc2\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:25.453763 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.453730    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/204a06bb-dc05-44d9-b667-69cbbf537982-maas-api-tls\") pod \"maas-api-e2e-derive-f6197270-79997b9d58-5qbc2\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:25.454389 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.454337    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/204a06bb-dc05-44d9-b667-69cbbf537982-openshift-service-ca\") pod \"maas-api-e2e-derive-f6197270-79997b9d58-5qbc2\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:25.456163 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.456142    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/204a06bb-dc05-44d9-b667-69cbbf537982-maas-api-tls\") pod \"maas-api-e2e-derive-f6197270-79997b9d58-5qbc2\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:25.469121 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.469100    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8wtt\" (UniqueName: \"kubernetes.io/projected/204a06bb-dc05-44d9-b667-69cbbf537982-kube-api-access-q8wtt\") pod \"maas-api-e2e-derive-f6197270-79997b9d58-5qbc2\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:25.643734 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.643708    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:25.758022 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.757985    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:25.759285 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.759258    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:25.777225 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.777154    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"]
Jul 15 04:36:25.781179 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:36:25.781153    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod204a06bb_dc05_44d9_b667_69cbbf537982.slice/crio-edd58e072b8ddd8a7fefa0745a57d241e835bd689e7462680a0681484a16b4ac WatchSource:0}: Error finding container edd58e072b8ddd8a7fefa0745a57d241e835bd689e7462680a0681484a16b4ac: Status 404 returned error can't find the container with id edd58e072b8ddd8a7fefa0745a57d241e835bd689e7462680a0681484a16b4ac
Jul 15 04:36:25.782693 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:25.782662    2524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 04:36:26.764013 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:26.763976    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2" event={"ID":"204a06bb-dc05-44d9-b667-69cbbf537982","Type":"ContainerStarted","Data":"6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b"}
Jul 15 04:36:26.764432 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:26.764028    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2" event={"ID":"204a06bb-dc05-44d9-b667-69cbbf537982","Type":"ContainerStarted","Data":"edd58e072b8ddd8a7fefa0745a57d241e835bd689e7462680a0681484a16b4ac"}
Jul 15 04:36:26.764432 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:26.764100    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:26.779938 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:26.779892    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2" podStartSLOduration=1.376508501 podStartE2EDuration="1.779876178s" totalImagesPullingTime="403.367677ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:36:26.779843623 +0000 UTC m=+1816.091224870" watchObservedRunningTime="2026-07-15 04:36:26.779876178 +0000 UTC m=+1816.091257424"
Jul 15 04:36:29.086588 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.086500    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:36:29.184486 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.184456    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-credential-socket\") pod \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") "
Jul 15 04:36:29.184486 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.184496    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-data\") pod \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") "
Jul 15 04:36:29.184790 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.184525    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8c9x2\" (UniqueName: \"kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-kube-api-access-8c9x2\") pod \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") "
Jul 15 04:36:29.184790 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.184583    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-token\") pod \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") "
Jul 15 04:36:29.184790 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.184643    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-envoy\") pod \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") "
Jul 15 04:36:29.184790 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.184666    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-socket\") pod \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") "
Jul 15 04:36:29.184790 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.184688    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istiod-ca-cert\") pod \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") "
Jul 15 04:36:29.184790 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.184713    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-podinfo\") pod \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") "
Jul 15 04:36:29.184790 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.184739    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-certs\") pod \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\" (UID: \"bc89b67d-1a23-4597-ad33-f7b9c15423f6\") "
Jul 15 04:36:29.185577 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.185305    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bc89b67d-1a23-4597-ad33-f7b9c15423f6" (UID: "bc89b67d-1a23-4597-ad33-f7b9c15423f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:36:29.185947 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.185912    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bc89b67d-1a23-4597-ad33-f7b9c15423f6" (UID: "bc89b67d-1a23-4597-ad33-f7b9c15423f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:36:29.186549 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.186377    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bc89b67d-1a23-4597-ad33-f7b9c15423f6" (UID: "bc89b67d-1a23-4597-ad33-f7b9c15423f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:36:29.186549 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.186514    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bc89b67d-1a23-4597-ad33-f7b9c15423f6" (UID: "bc89b67d-1a23-4597-ad33-f7b9c15423f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:36:29.187862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.187750    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-data" (OuterVolumeSpecName: "istio-data") pod "bc89b67d-1a23-4597-ad33-f7b9c15423f6" (UID: "bc89b67d-1a23-4597-ad33-f7b9c15423f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:36:29.189019 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.188985    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-kube-api-access-8c9x2" (OuterVolumeSpecName: "kube-api-access-8c9x2") pod "bc89b67d-1a23-4597-ad33-f7b9c15423f6" (UID: "bc89b67d-1a23-4597-ad33-f7b9c15423f6"). InnerVolumeSpecName "kube-api-access-8c9x2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:36:29.189200 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.189168    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bc89b67d-1a23-4597-ad33-f7b9c15423f6" (UID: "bc89b67d-1a23-4597-ad33-f7b9c15423f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:36:29.189636 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.189584    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bc89b67d-1a23-4597-ad33-f7b9c15423f6" (UID: "bc89b67d-1a23-4597-ad33-f7b9c15423f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:36:29.189750 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.189736    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-token" (OuterVolumeSpecName: "istio-token") pod "bc89b67d-1a23-4597-ad33-f7b9c15423f6" (UID: "bc89b67d-1a23-4597-ad33-f7b9c15423f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:36:29.286362 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.286321    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:29.286362 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.286352    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:29.286362 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.286363    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:29.286362 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.286372    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:29.286651 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.286385    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:29.286651 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.286398    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:29.286651 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.286406    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:29.286651 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.286414    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc89b67d-1a23-4597-ad33-f7b9c15423f6-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:29.286651 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.286423    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8c9x2\" (UniqueName: \"kubernetes.io/projected/bc89b67d-1a23-4597-ad33-f7b9c15423f6-kube-api-access-8c9x2\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:36:29.783872 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.783744    2524 generic.go:356] "Generic (PLEG): container finished" podID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerID="9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec" exitCode=0
Jul 15 04:36:29.783872 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.783816    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"
Jul 15 04:36:29.784130 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.783815    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" event={"ID":"bc89b67d-1a23-4597-ad33-f7b9c15423f6","Type":"ContainerDied","Data":"9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec"}
Jul 15 04:36:29.784130 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.783970    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2" event={"ID":"bc89b67d-1a23-4597-ad33-f7b9c15423f6","Type":"ContainerDied","Data":"7a7d5947b95d670a1c468f6c3b0ba3c4233c361325c108d3645aa7d5e3203c6a"}
Jul 15 04:36:29.784130 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.784007    2524 scope.go:117] "RemoveContainer" containerID="9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec"
Jul 15 04:36:29.793969 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.793944    2524 scope.go:117] "RemoveContainer" containerID="fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95"
Jul 15 04:36:29.802673 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.802645    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"]
Jul 15 04:36:29.803523 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.803504    2524 scope.go:117] "RemoveContainer" containerID="9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec"
Jul 15 04:36:29.803874 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:36:29.803854    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec\": container with ID starting with 9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec not found: ID does not exist" containerID="9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec"
Jul 15 04:36:29.803932 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.803884    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec"} err="failed to get container status \"9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec\": rpc error: code = NotFound desc = could not find container \"9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec\": container with ID starting with 9b9afe205e6ee7dd5079cab86db1d9fc90a740b20b3660d46475523c010522ec not found: ID does not exist"
Jul 15 04:36:29.803932 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.803904    2524 scope.go:117] "RemoveContainer" containerID="fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95"
Jul 15 04:36:29.804147 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:36:29.804131    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95\": container with ID starting with fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95 not found: ID does not exist" containerID="fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95"
Jul 15 04:36:29.804193 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.804154    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95"} err="failed to get container status \"fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95\": rpc error: code = NotFound desc = could not find container \"fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95\": container with ID starting with fc9aa1f7b24c815a57516a60351c682ae9a64115f7a8982d9de7dbf3d15c6d95 not found: ID does not exist"
Jul 15 04:36:29.808302 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:29.808275    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c27a526e-openshift-default-589d8d5557-vtrq2"]
Jul 15 04:36:31.432575 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:31.432535    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" path="/var/lib/kubelet/pods/bc89b67d-1a23-4597-ad33-f7b9c15423f6/volumes"
Jul 15 04:36:32.782895 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:32.782865    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:36:35.813918 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:35.813885    2524 generic.go:356] "Generic (PLEG): container finished" podID="7913969e-b095-47dd-8fa5-48d151806141" containerID="acea53cc00afade44b9ccf88906eec0ffa9d4655a68a102a2f658d5b9a70a424" exitCode=0
Jul 15 04:36:35.814373 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:35.813967    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" event={"ID":"7913969e-b095-47dd-8fa5-48d151806141","Type":"ContainerDied","Data":"acea53cc00afade44b9ccf88906eec0ffa9d4655a68a102a2f658d5b9a70a424"}
Jul 15 04:36:35.814479 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:35.814462    2524 scope.go:117] "RemoveContainer" containerID="acea53cc00afade44b9ccf88906eec0ffa9d4655a68a102a2f658d5b9a70a424"
Jul 15 04:36:35.815525 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:35.815497    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:36:35.815621 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:35.815565    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:36:35.815674 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:35.815620    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:36:36.821017 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:36.820980    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" event={"ID":"7913969e-b095-47dd-8fa5-48d151806141","Type":"ContainerStarted","Data":"0c80bc1f0a15794c9a6e3ee052a3f11cf7e362f098245c40dc94bf9de8705783"}
Jul 15 04:36:37.422938 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:37.422900    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:37.423961 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:37.423939    2524 patch_prober.go:28] interesting pod/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 15 04:36:37.424077 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:37.423986    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 15 04:36:38.423228 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:38.423192    2524 patch_prober.go:28] interesting pod/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 15 04:36:38.423670 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:38.423267    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 15 04:36:39.422950 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:39.422920    2524 patch_prober.go:28] interesting pod/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 15 04:36:39.423260 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:39.423234    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 15 04:36:40.422752 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:40.422716    2524 patch_prober.go:28] interesting pod/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 15 04:36:40.422959 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:40.422768    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 15 04:36:40.758987 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:40.758895    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:41.432257 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:41.432230    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:36:41.433137 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:36:41.433121    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:37:40.121174 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.121137    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"]
Jul 15 04:37:40.121775 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.121354    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2" podUID="204a06bb-dc05-44d9-b667-69cbbf537982" containerName="maas-api" containerID="cri-o://6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b" gracePeriod=30
Jul 15 04:37:40.367193 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.367167    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:37:40.399418 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.399357    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/204a06bb-dc05-44d9-b667-69cbbf537982-maas-api-tls\") pod \"204a06bb-dc05-44d9-b667-69cbbf537982\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") "
Jul 15 04:37:40.399418 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.399403    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8wtt\" (UniqueName: \"kubernetes.io/projected/204a06bb-dc05-44d9-b667-69cbbf537982-kube-api-access-q8wtt\") pod \"204a06bb-dc05-44d9-b667-69cbbf537982\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") "
Jul 15 04:37:40.399620 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.399451    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/204a06bb-dc05-44d9-b667-69cbbf537982-openshift-service-ca\") pod \"204a06bb-dc05-44d9-b667-69cbbf537982\" (UID: \"204a06bb-dc05-44d9-b667-69cbbf537982\") "
Jul 15 04:37:40.399881 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.399856    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/204a06bb-dc05-44d9-b667-69cbbf537982-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "204a06bb-dc05-44d9-b667-69cbbf537982" (UID: "204a06bb-dc05-44d9-b667-69cbbf537982"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:37:40.401647 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.401615    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/204a06bb-dc05-44d9-b667-69cbbf537982-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "204a06bb-dc05-44d9-b667-69cbbf537982" (UID: "204a06bb-dc05-44d9-b667-69cbbf537982"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:37:40.401755 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.401682    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/204a06bb-dc05-44d9-b667-69cbbf537982-kube-api-access-q8wtt" (OuterVolumeSpecName: "kube-api-access-q8wtt") pod "204a06bb-dc05-44d9-b667-69cbbf537982" (UID: "204a06bb-dc05-44d9-b667-69cbbf537982"). InnerVolumeSpecName "kube-api-access-q8wtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:37:40.501531 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.501502    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/204a06bb-dc05-44d9-b667-69cbbf537982-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:37:40.501531 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.501535    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8wtt\" (UniqueName: \"kubernetes.io/projected/204a06bb-dc05-44d9-b667-69cbbf537982-kube-api-access-q8wtt\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:37:40.501786 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:40.501551    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/204a06bb-dc05-44d9-b667-69cbbf537982-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:37:41.114331 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:41.114296    2524 generic.go:356] "Generic (PLEG): container finished" podID="204a06bb-dc05-44d9-b667-69cbbf537982" containerID="6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b" exitCode=0
Jul 15 04:37:41.114513 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:41.114346    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"
Jul 15 04:37:41.114513 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:41.114353    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2" event={"ID":"204a06bb-dc05-44d9-b667-69cbbf537982","Type":"ContainerDied","Data":"6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b"}
Jul 15 04:37:41.114513 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:41.114429    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2" event={"ID":"204a06bb-dc05-44d9-b667-69cbbf537982","Type":"ContainerDied","Data":"edd58e072b8ddd8a7fefa0745a57d241e835bd689e7462680a0681484a16b4ac"}
Jul 15 04:37:41.114513 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:41.114453    2524 scope.go:117] "RemoveContainer" containerID="6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b"
Jul 15 04:37:41.123838 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:41.123672    2524 scope.go:117] "RemoveContainer" containerID="6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b"
Jul 15 04:37:41.124036 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:37:41.123916    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b\": container with ID starting with 6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b not found: ID does not exist" containerID="6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b"
Jul 15 04:37:41.124036 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:41.123936    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b"} err="failed to get container status \"6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b\": rpc error: code = NotFound desc = could not find container \"6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b\": container with ID starting with 6ae1e3368f553aa4d37285dee995b20c8d20f29e07adaf860954cc85015b520b not found: ID does not exist"
Jul 15 04:37:41.135276 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:41.135230    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"]
Jul 15 04:37:41.137014 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:41.136995    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f6197270-79997b9d58-5qbc2"]
Jul 15 04:37:41.432287 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:41.432250    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="204a06bb-dc05-44d9-b667-69cbbf537982" path="/var/lib/kubelet/pods/204a06bb-dc05-44d9-b667-69cbbf537982/volumes"
Jul 15 04:37:54.965359 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:54.965320    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"]
Jul 15 04:37:54.965993 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:54.965610    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy" containerID="cri-o://0c80bc1f0a15794c9a6e3ee052a3f11cf7e362f098245c40dc94bf9de8705783" gracePeriod=30
Jul 15 04:37:56.434159 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:56.434117    2524 patch_prober.go:28] interesting pod/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 04:37:56.434517 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:37:56.434192    2524 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 04:38:00.207931 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.207893    2524 generic.go:356] "Generic (PLEG): container finished" podID="7913969e-b095-47dd-8fa5-48d151806141" containerID="0c80bc1f0a15794c9a6e3ee052a3f11cf7e362f098245c40dc94bf9de8705783" exitCode=0
Jul 15 04:38:00.208318 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.207968    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" event={"ID":"7913969e-b095-47dd-8fa5-48d151806141","Type":"ContainerDied","Data":"0c80bc1f0a15794c9a6e3ee052a3f11cf7e362f098245c40dc94bf9de8705783"}
Jul 15 04:38:00.208318 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.208040    2524 scope.go:117] "RemoveContainer" containerID="acea53cc00afade44b9ccf88906eec0ffa9d4655a68a102a2f658d5b9a70a424"
Jul 15 04:38:00.244069 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.244042    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:38:00.390745 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.390715    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-socket\") pod \"7913969e-b095-47dd-8fa5-48d151806141\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") "
Jul 15 04:38:00.390915 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.390758    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9kww\" (UniqueName: \"kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-kube-api-access-z9kww\") pod \"7913969e-b095-47dd-8fa5-48d151806141\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") "
Jul 15 04:38:00.390915 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.390776    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-envoy\") pod \"7913969e-b095-47dd-8fa5-48d151806141\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") "
Jul 15 04:38:00.390915 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.390793    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-certs\") pod \"7913969e-b095-47dd-8fa5-48d151806141\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") "
Jul 15 04:38:00.390915 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.390841    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-data\") pod \"7913969e-b095-47dd-8fa5-48d151806141\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") "
Jul 15 04:38:00.390915 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.390890    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-istio-token\") pod \"7913969e-b095-47dd-8fa5-48d151806141\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") "
Jul 15 04:38:00.391172 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.390918    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-credential-socket\") pod \"7913969e-b095-47dd-8fa5-48d151806141\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") "
Jul 15 04:38:00.391172 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.390945    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7913969e-b095-47dd-8fa5-48d151806141-istiod-ca-cert\") pod \"7913969e-b095-47dd-8fa5-48d151806141\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") "
Jul 15 04:38:00.391172 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.390970    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7913969e-b095-47dd-8fa5-48d151806141-istio-podinfo\") pod \"7913969e-b095-47dd-8fa5-48d151806141\" (UID: \"7913969e-b095-47dd-8fa5-48d151806141\") "
Jul 15 04:38:00.391338 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.391189    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7913969e-b095-47dd-8fa5-48d151806141" (UID: "7913969e-b095-47dd-8fa5-48d151806141"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:38:00.391338 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.391200    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7913969e-b095-47dd-8fa5-48d151806141" (UID: "7913969e-b095-47dd-8fa5-48d151806141"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:38:00.391338 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.391303    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:38:00.391338 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.391323    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:38:00.391541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.391426    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7913969e-b095-47dd-8fa5-48d151806141" (UID: "7913969e-b095-47dd-8fa5-48d151806141"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:38:00.391541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.391500    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7913969e-b095-47dd-8fa5-48d151806141-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7913969e-b095-47dd-8fa5-48d151806141" (UID: "7913969e-b095-47dd-8fa5-48d151806141"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:38:00.392161 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.392127    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-data" (OuterVolumeSpecName: "istio-data") pod "7913969e-b095-47dd-8fa5-48d151806141" (UID: "7913969e-b095-47dd-8fa5-48d151806141"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:38:00.393146 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.393117    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7913969e-b095-47dd-8fa5-48d151806141" (UID: "7913969e-b095-47dd-8fa5-48d151806141"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:38:00.393252 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.393199    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-kube-api-access-z9kww" (OuterVolumeSpecName: "kube-api-access-z9kww") pod "7913969e-b095-47dd-8fa5-48d151806141" (UID: "7913969e-b095-47dd-8fa5-48d151806141"). InnerVolumeSpecName "kube-api-access-z9kww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:38:00.393252 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.393209    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-istio-token" (OuterVolumeSpecName: "istio-token") pod "7913969e-b095-47dd-8fa5-48d151806141" (UID: "7913969e-b095-47dd-8fa5-48d151806141"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:38:00.393491 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.393470    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7913969e-b095-47dd-8fa5-48d151806141-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7913969e-b095-47dd-8fa5-48d151806141" (UID: "7913969e-b095-47dd-8fa5-48d151806141"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:38:00.492064 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.491986    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9kww\" (UniqueName: \"kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-kube-api-access-z9kww\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:38:00.492064 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.492013    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:38:00.492064 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.492022    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:38:00.492064 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.492030    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7913969e-b095-47dd-8fa5-48d151806141-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:38:00.492064 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.492038    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7913969e-b095-47dd-8fa5-48d151806141-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:38:00.492064 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.492046    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7913969e-b095-47dd-8fa5-48d151806141-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:38:00.492064 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:00.492053    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7913969e-b095-47dd-8fa5-48d151806141-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:38:01.214470 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:01.214435    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx" event={"ID":"7913969e-b095-47dd-8fa5-48d151806141","Type":"ContainerDied","Data":"22c10f7c44ff19f81a5da1e9904075f39841cc0e3b61e2c74e5af2129e3cdc01"}
Jul 15 04:38:01.214968 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:01.214473    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"
Jul 15 04:38:01.214968 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:01.214495    2524 scope.go:117] "RemoveContainer" containerID="0c80bc1f0a15794c9a6e3ee052a3f11cf7e362f098245c40dc94bf9de8705783"
Jul 15 04:38:01.238913 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:01.238880    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"]
Jul 15 04:38:01.247471 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:01.247446    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-f6197270-openshift-default-5f6885cfb7-l2rfx"]
Jul 15 04:38:01.432423 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:38:01.432387    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7913969e-b095-47dd-8fa5-48d151806141" path="/var/lib/kubelet/pods/7913969e-b095-47dd-8fa5-48d151806141/volumes"
Jul 15 04:39:30.178473 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178400    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"]
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178841    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="204a06bb-dc05-44d9-b667-69cbbf537982" containerName="maas-api"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178855    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="204a06bb-dc05-44d9-b667-69cbbf537982" containerName="maas-api"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178866    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178871    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178879    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178886    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178891    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178896    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178953    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="204a06bb-dc05-44d9-b667-69cbbf537982" containerName="maas-api"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178961    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178967    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc89b67d-1a23-4597-ad33-f7b9c15423f6" containerName="istio-proxy"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178974    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy"
Jul 15 04:39:30.179011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.178979    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy"
Jul 15 04:39:30.182022 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.182004    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.184507 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.184484    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d959dbe7-openshift-default-dockercfg-tv4bm\""
Jul 15 04:39:30.201453 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.201426    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"]
Jul 15 04:39:30.257709 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.257677    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-data\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.257856 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.257725    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-credential-socket\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.257856 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.257742    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e42d141b-79e3-48d6-8dda-37fccae16390-istio-podinfo\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.257856 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.257759    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-socket\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.257856 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.257804    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-certs\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.257856 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.257843    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e42d141b-79e3-48d6-8dda-37fccae16390-istiod-ca-cert\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.258061 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.257864    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-envoy\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.258061 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.257881    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-istio-token\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.258061 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.257900    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcn2m\" (UniqueName: \"kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-kube-api-access-vcn2m\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.358707 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.358669    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-data\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.358868 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.358725    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-credential-socket\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.358868 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.358746    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e42d141b-79e3-48d6-8dda-37fccae16390-istio-podinfo\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.358868 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.358772    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-socket\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.358868 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.358794    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-certs\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.358868 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.358827    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e42d141b-79e3-48d6-8dda-37fccae16390-istiod-ca-cert\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.358868 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.358851    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-envoy\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.359187 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.359125    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-istio-token\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.359187 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.359160    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-socket\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.359294 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.359186    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-data\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.359294 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.359166    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcn2m\" (UniqueName: \"kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-kube-api-access-vcn2m\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.359407 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.359323    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-certs\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.359683 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.359629    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-credential-socket\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.359683 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.359700    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e42d141b-79e3-48d6-8dda-37fccae16390-istiod-ca-cert\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.361347 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.361327    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-envoy\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.361480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.361462    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e42d141b-79e3-48d6-8dda-37fccae16390-istio-podinfo\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.368174 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.368153    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-istio-token\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.368463 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.368440    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcn2m\" (UniqueName: \"kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-kube-api-access-vcn2m\") pod \"e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") " pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.493563 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.493469    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:30.632908 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.632879    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"]
Jul 15 04:39:30.634589 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:39:30.634564    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42d141b_79e3_48d6_8dda_37fccae16390.slice/crio-7cc436e1665654d68f1c995c71e8324aac44af9a77a1802e4e8c267734342965 WatchSource:0}: Error finding container 7cc436e1665654d68f1c995c71e8324aac44af9a77a1802e4e8c267734342965: Status 404 returned error can't find the container with id 7cc436e1665654d68f1c995c71e8324aac44af9a77a1802e4e8c267734342965
Jul 15 04:39:30.636578 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.636531    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:39:30.636701 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.636678    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:39:30.636750 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:30.636731    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:39:31.628824 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:31.628791    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" event={"ID":"e42d141b-79e3-48d6-8dda-37fccae16390","Type":"ContainerStarted","Data":"ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f"}
Jul 15 04:39:31.629299 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:31.628851    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" event={"ID":"e42d141b-79e3-48d6-8dda-37fccae16390","Type":"ContainerStarted","Data":"7cc436e1665654d68f1c995c71e8324aac44af9a77a1802e4e8c267734342965"}
Jul 15 04:39:31.650006 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:31.649952    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" podStartSLOduration=1.649936938 podStartE2EDuration="1.649936938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:39:31.64832281 +0000 UTC m=+2000.959704059" watchObservedRunningTime="2026-07-15 04:39:31.649936938 +0000 UTC m=+2000.961318184"
Jul 15 04:39:32.494682 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:32.494654    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:32.499303 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:32.499279    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:32.633895 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:32.633859    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:32.634834 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:32.634816    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:33.825351 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.825318    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"]
Jul 15 04:39:33.825901 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.825883    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy"
Jul 15 04:39:33.825961 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.825903    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="7913969e-b095-47dd-8fa5-48d151806141" containerName="istio-proxy"
Jul 15 04:39:33.829280 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.829261    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:33.831940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.831917    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d959dbe7-serving-cert\""
Jul 15 04:39:33.846313 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.846288    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"]
Jul 15 04:39:33.890077 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.890049    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f545288c-fba0-48ed-9cf3-9249d32c8033-maas-api-tls\") pod \"maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:33.890255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.890082    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f545288c-fba0-48ed-9cf3-9249d32c8033-openshift-service-ca\") pod \"maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:33.890255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.890126    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxkfm\" (UniqueName: \"kubernetes.io/projected/f545288c-fba0-48ed-9cf3-9249d32c8033-kube-api-access-pxkfm\") pod \"maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:33.991437 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.991403    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxkfm\" (UniqueName: \"kubernetes.io/projected/f545288c-fba0-48ed-9cf3-9249d32c8033-kube-api-access-pxkfm\") pod \"maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:33.991642 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.991485    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f545288c-fba0-48ed-9cf3-9249d32c8033-maas-api-tls\") pod \"maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:33.991642 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.991504    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f545288c-fba0-48ed-9cf3-9249d32c8033-openshift-service-ca\") pod \"maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:33.992110 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.992083    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f545288c-fba0-48ed-9cf3-9249d32c8033-openshift-service-ca\") pod \"maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:33.994128 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:33.994096    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f545288c-fba0-48ed-9cf3-9249d32c8033-maas-api-tls\") pod \"maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:34.003422 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:34.003403    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxkfm\" (UniqueName: \"kubernetes.io/projected/f545288c-fba0-48ed-9cf3-9249d32c8033-kube-api-access-pxkfm\") pod \"maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:34.142725 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:34.142699    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:34.281335 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:34.281310    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"]
Jul 15 04:39:34.282357 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:39:34.282327    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf545288c_fba0_48ed_9cf3_9249d32c8033.slice/crio-929509e22d4d9b7c100d47eff17e26cf85ffe0c7fe9f901fee099194cbbaf28d WatchSource:0}: Error finding container 929509e22d4d9b7c100d47eff17e26cf85ffe0c7fe9f901fee099194cbbaf28d: Status 404 returned error can't find the container with id 929509e22d4d9b7c100d47eff17e26cf85ffe0c7fe9f901fee099194cbbaf28d
Jul 15 04:39:34.642472 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:34.642442    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k" event={"ID":"f545288c-fba0-48ed-9cf3-9249d32c8033","Type":"ContainerStarted","Data":"929509e22d4d9b7c100d47eff17e26cf85ffe0c7fe9f901fee099194cbbaf28d"}
Jul 15 04:39:35.648706 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:35.648661    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k" event={"ID":"f545288c-fba0-48ed-9cf3-9249d32c8033","Type":"ContainerStarted","Data":"d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647"}
Jul 15 04:39:35.649168 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:35.648776    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:35.664535 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:35.664486    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k" podStartSLOduration=2.203218718 podStartE2EDuration="2.664471352s" totalImagesPullingTime="461.252634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:39:35.663743388 +0000 UTC m=+2004.975124646" watchObservedRunningTime="2026-07-15 04:39:35.664471352 +0000 UTC m=+2004.975852620"
Jul 15 04:39:41.665892 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:41.665860    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:46.711569 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:46.711463    2524 generic.go:356] "Generic (PLEG): container finished" podID="e42d141b-79e3-48d6-8dda-37fccae16390" containerID="ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f" exitCode=0
Jul 15 04:39:46.711569 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:46.711533    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" event={"ID":"e42d141b-79e3-48d6-8dda-37fccae16390","Type":"ContainerDied","Data":"ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f"}
Jul 15 04:39:46.712020 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:46.711914    2524 scope.go:117] "RemoveContainer" containerID="ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f"
Jul 15 04:39:46.713037 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:46.713005    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:39:46.713112 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:46.713095    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:39:46.713164 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:46.713149    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:39:47.634819 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:47.634788    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:47.720400 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:47.720358    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" event={"ID":"e42d141b-79e3-48d6-8dda-37fccae16390","Type":"ContainerStarted","Data":"e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b"}
Jul 15 04:39:48.493923 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:48.493886    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:48.495132 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:48.495104    2524 patch_prober.go:28] interesting pod/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 15 04:39:48.495262 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:48.495156    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 15 04:39:49.494421 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:49.494381    2524 patch_prober.go:28] interesting pod/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 15 04:39:49.494859 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:49.494440    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 15 04:39:50.494072 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:50.494029    2524 patch_prober.go:28] interesting pod/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 15 04:39:50.494261 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:50.494087    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 15 04:39:51.494008 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:51.493973    2524 patch_prober.go:28] interesting pod/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 15 04:39:51.494414 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:51.494035    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 15 04:39:52.499147 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:52.499114    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:52.499684 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:52.499379    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:52.500590 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:52.500571    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:39:55.696276 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:55.696241    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"]
Jul 15 04:39:55.697050 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:55.696988    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k" podUID="f545288c-fba0-48ed-9cf3-9249d32c8033" containerName="maas-api" containerID="cri-o://d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647" gracePeriod=30
Jul 15 04:39:55.961246 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:55.961222    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:56.012792 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.012762    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxkfm\" (UniqueName: \"kubernetes.io/projected/f545288c-fba0-48ed-9cf3-9249d32c8033-kube-api-access-pxkfm\") pod \"f545288c-fba0-48ed-9cf3-9249d32c8033\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") "
Jul 15 04:39:56.012935 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.012817    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f545288c-fba0-48ed-9cf3-9249d32c8033-openshift-service-ca\") pod \"f545288c-fba0-48ed-9cf3-9249d32c8033\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") "
Jul 15 04:39:56.012935 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.012858    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f545288c-fba0-48ed-9cf3-9249d32c8033-maas-api-tls\") pod \"f545288c-fba0-48ed-9cf3-9249d32c8033\" (UID: \"f545288c-fba0-48ed-9cf3-9249d32c8033\") "
Jul 15 04:39:56.013210 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.013185    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f545288c-fba0-48ed-9cf3-9249d32c8033-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f545288c-fba0-48ed-9cf3-9249d32c8033" (UID: "f545288c-fba0-48ed-9cf3-9249d32c8033"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:39:56.015053 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.015026    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f545288c-fba0-48ed-9cf3-9249d32c8033-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f545288c-fba0-48ed-9cf3-9249d32c8033" (UID: "f545288c-fba0-48ed-9cf3-9249d32c8033"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:39:56.015053 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.015025    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f545288c-fba0-48ed-9cf3-9249d32c8033-kube-api-access-pxkfm" (OuterVolumeSpecName: "kube-api-access-pxkfm") pod "f545288c-fba0-48ed-9cf3-9249d32c8033" (UID: "f545288c-fba0-48ed-9cf3-9249d32c8033"). InnerVolumeSpecName "kube-api-access-pxkfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:39:56.113706 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.113670    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f545288c-fba0-48ed-9cf3-9249d32c8033-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:39:56.113706 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.113701    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f545288c-fba0-48ed-9cf3-9249d32c8033-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:39:56.113706 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.113710    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxkfm\" (UniqueName: \"kubernetes.io/projected/f545288c-fba0-48ed-9cf3-9249d32c8033-kube-api-access-pxkfm\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:39:56.764220 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.764180    2524 generic.go:356] "Generic (PLEG): container finished" podID="f545288c-fba0-48ed-9cf3-9249d32c8033" containerID="d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647" exitCode=0
Jul 15 04:39:56.764653 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.764225    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"
Jul 15 04:39:56.764653 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.764270    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k" event={"ID":"f545288c-fba0-48ed-9cf3-9249d32c8033","Type":"ContainerDied","Data":"d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647"}
Jul 15 04:39:56.764653 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.764329    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k" event={"ID":"f545288c-fba0-48ed-9cf3-9249d32c8033","Type":"ContainerDied","Data":"929509e22d4d9b7c100d47eff17e26cf85ffe0c7fe9f901fee099194cbbaf28d"}
Jul 15 04:39:56.764653 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.764353    2524 scope.go:117] "RemoveContainer" containerID="d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647"
Jul 15 04:39:56.773959 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.773941    2524 scope.go:117] "RemoveContainer" containerID="d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647"
Jul 15 04:39:56.774210 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:39:56.774192    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647\": container with ID starting with d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647 not found: ID does not exist" containerID="d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647"
Jul 15 04:39:56.774249 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.774219    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647"} err="failed to get container status \"d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647\": rpc error: code = NotFound desc = could not find container \"d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647\": container with ID starting with d95f459f167e3868494c1190fccf88305ec013c3a49f03869921c4b8203f6647 not found: ID does not exist"
Jul 15 04:39:56.785109 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.785085    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"]
Jul 15 04:39:56.789215 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:56.789196    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d959dbe7-8467c8875f-2n96k"]
Jul 15 04:39:57.432262 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:39:57.432226    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f545288c-fba0-48ed-9cf3-9249d32c8033" path="/var/lib/kubelet/pods/f545288c-fba0-48ed-9cf3-9249d32c8033/volumes"
Jul 15 04:40:11.079471 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:11.079427    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"]
Jul 15 04:40:11.080018 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:11.079682    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" containerName="istio-proxy" containerID="cri-o://e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b" gracePeriod=30
Jul 15 04:40:16.337284 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.337257    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:40:16.404427 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.404387    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-certs\") pod \"e42d141b-79e3-48d6-8dda-37fccae16390\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") "
Jul 15 04:40:16.404683 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.404445    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-socket\") pod \"e42d141b-79e3-48d6-8dda-37fccae16390\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") "
Jul 15 04:40:16.404683 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.404465    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-istio-token\") pod \"e42d141b-79e3-48d6-8dda-37fccae16390\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") "
Jul 15 04:40:16.404683 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.404485    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcn2m\" (UniqueName: \"kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-kube-api-access-vcn2m\") pod \"e42d141b-79e3-48d6-8dda-37fccae16390\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") "
Jul 15 04:40:16.404683 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.404632    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-credential-socket\") pod \"e42d141b-79e3-48d6-8dda-37fccae16390\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") "
Jul 15 04:40:16.404683 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.404685    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e42d141b-79e3-48d6-8dda-37fccae16390-istio-podinfo\") pod \"e42d141b-79e3-48d6-8dda-37fccae16390\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") "
Jul 15 04:40:16.404984 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.404706    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e42d141b-79e3-48d6-8dda-37fccae16390-istiod-ca-cert\") pod \"e42d141b-79e3-48d6-8dda-37fccae16390\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") "
Jul 15 04:40:16.404984 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.404727    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-envoy\") pod \"e42d141b-79e3-48d6-8dda-37fccae16390\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") "
Jul 15 04:40:16.404984 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.404759    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e42d141b-79e3-48d6-8dda-37fccae16390" (UID: "e42d141b-79e3-48d6-8dda-37fccae16390"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:40:16.404984 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.404836    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-data\") pod \"e42d141b-79e3-48d6-8dda-37fccae16390\" (UID: \"e42d141b-79e3-48d6-8dda-37fccae16390\") "
Jul 15 04:40:16.405207 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.405177    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:40:16.405207 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.404831    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e42d141b-79e3-48d6-8dda-37fccae16390" (UID: "e42d141b-79e3-48d6-8dda-37fccae16390"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:40:16.405312 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.405226    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e42d141b-79e3-48d6-8dda-37fccae16390-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e42d141b-79e3-48d6-8dda-37fccae16390" (UID: "e42d141b-79e3-48d6-8dda-37fccae16390"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:40:16.405525 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.405455    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e42d141b-79e3-48d6-8dda-37fccae16390" (UID: "e42d141b-79e3-48d6-8dda-37fccae16390"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:40:16.406036 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.406009    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-data" (OuterVolumeSpecName: "istio-data") pod "e42d141b-79e3-48d6-8dda-37fccae16390" (UID: "e42d141b-79e3-48d6-8dda-37fccae16390"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:40:16.407036 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.407009    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-istio-token" (OuterVolumeSpecName: "istio-token") pod "e42d141b-79e3-48d6-8dda-37fccae16390" (UID: "e42d141b-79e3-48d6-8dda-37fccae16390"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:40:16.407137 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.407079    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-kube-api-access-vcn2m" (OuterVolumeSpecName: "kube-api-access-vcn2m") pod "e42d141b-79e3-48d6-8dda-37fccae16390" (UID: "e42d141b-79e3-48d6-8dda-37fccae16390"). InnerVolumeSpecName "kube-api-access-vcn2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:40:16.407137 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.407126    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e42d141b-79e3-48d6-8dda-37fccae16390" (UID: "e42d141b-79e3-48d6-8dda-37fccae16390"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:40:16.407217 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.407136    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e42d141b-79e3-48d6-8dda-37fccae16390-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e42d141b-79e3-48d6-8dda-37fccae16390" (UID: "e42d141b-79e3-48d6-8dda-37fccae16390"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:40:16.506308 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.506265    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:40:16.506308 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.506299    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:40:16.506308 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.506308    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:40:16.506308 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.506317    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcn2m\" (UniqueName: \"kubernetes.io/projected/e42d141b-79e3-48d6-8dda-37fccae16390-kube-api-access-vcn2m\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:40:16.506647 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.506326    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:40:16.506647 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.506334    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e42d141b-79e3-48d6-8dda-37fccae16390-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:40:16.506647 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.506342    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e42d141b-79e3-48d6-8dda-37fccae16390-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:40:16.506647 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.506350    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e42d141b-79e3-48d6-8dda-37fccae16390-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:40:16.860050 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.859950    2524 generic.go:356] "Generic (PLEG): container finished" podID="e42d141b-79e3-48d6-8dda-37fccae16390" containerID="e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b" exitCode=0
Jul 15 04:40:16.860050 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.860007    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"
Jul 15 04:40:16.860050 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.860017    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" event={"ID":"e42d141b-79e3-48d6-8dda-37fccae16390","Type":"ContainerDied","Data":"e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b"}
Jul 15 04:40:16.860536 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.860085    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m" event={"ID":"e42d141b-79e3-48d6-8dda-37fccae16390","Type":"ContainerDied","Data":"7cc436e1665654d68f1c995c71e8324aac44af9a77a1802e4e8c267734342965"}
Jul 15 04:40:16.860536 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.860121    2524 scope.go:117] "RemoveContainer" containerID="e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b"
Jul 15 04:40:16.870415 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.870395    2524 scope.go:117] "RemoveContainer" containerID="ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f"
Jul 15 04:40:16.879241 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.879218    2524 scope.go:117] "RemoveContainer" containerID="e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b"
Jul 15 04:40:16.879554 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:40:16.879538    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b\": container with ID starting with e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b not found: ID does not exist" containerID="e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b"
Jul 15 04:40:16.879632 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.879564    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b"} err="failed to get container status \"e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b\": rpc error: code = NotFound desc = could not find container \"e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b\": container with ID starting with e67dfbb00b825f21a81d4d5c333bf09d9e9b622753a7915c3ed7f6bc768f667b not found: ID does not exist"
Jul 15 04:40:16.879632 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.879581    2524 scope.go:117] "RemoveContainer" containerID="ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f"
Jul 15 04:40:16.879861 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:40:16.879845    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f\": container with ID starting with ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f not found: ID does not exist" containerID="ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f"
Jul 15 04:40:16.879912 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.879864    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f"} err="failed to get container status \"ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f\": rpc error: code = NotFound desc = could not find container \"ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f\": container with ID starting with ad01e07b57de279232aaed2dc1f0531a8c2fd50c4dc26854a53f17951e085a7f not found: ID does not exist"
Jul 15 04:40:16.882869 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.882846    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"]
Jul 15 04:40:16.889970 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:16.889947    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d959dbe7-openshift-default-77c7fccf75-74q4m"]
Jul 15 04:40:16.902850 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:40:16.902821    2524 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42d141b_79e3_48d6_8dda_37fccae16390.slice/crio-7cc436e1665654d68f1c995c71e8324aac44af9a77a1802e4e8c267734342965\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42d141b_79e3_48d6_8dda_37fccae16390.slice\": RecentStats: unable to find data in memory cache]"
Jul 15 04:40:17.441004 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:17.440953    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" path="/var/lib/kubelet/pods/e42d141b-79e3-48d6-8dda-37fccae16390/volumes"
Jul 15 04:40:30.520935 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.520898    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"]
Jul 15 04:40:30.521727 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.521705    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f545288c-fba0-48ed-9cf3-9249d32c8033" containerName="maas-api"
Jul 15 04:40:30.521727 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.521729    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="f545288c-fba0-48ed-9cf3-9249d32c8033" containerName="maas-api"
Jul 15 04:40:30.521904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.521745    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" containerName="istio-proxy"
Jul 15 04:40:30.521904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.521753    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" containerName="istio-proxy"
Jul 15 04:40:30.521904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.521770    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" containerName="istio-proxy"
Jul 15 04:40:30.521904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.521778    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" containerName="istio-proxy"
Jul 15 04:40:30.522102 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.521926    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" containerName="istio-proxy"
Jul 15 04:40:30.522102 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.521941    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="e42d141b-79e3-48d6-8dda-37fccae16390" containerName="istio-proxy"
Jul 15 04:40:30.522102 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.521951    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="f545288c-fba0-48ed-9cf3-9249d32c8033" containerName="maas-api"
Jul 15 04:40:30.533133 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.533096    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.536243 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.536200    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-699a3596-openshift-default-dockercfg-5qkqx\""
Jul 15 04:40:30.538865 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.538835    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"]
Jul 15 04:40:30.635913 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.635873    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-certs\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.636097 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.635934    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-envoy\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.636097 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.635959    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-data\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.636097 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.635975    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-token\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.636097 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.636019    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979c607b-69fa-4ac6-ab3e-03c7550e4274-istiod-ca-cert\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.636097 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.636044    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-credential-socket\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.636097 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.636063    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-socket\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.636321 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.636115    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-podinfo\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.636321 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.636154    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49mkc\" (UniqueName: \"kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-kube-api-access-49mkc\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.736914 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.736874    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-certs\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737111 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.736927    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-envoy\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737111 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.736998    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-data\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737111 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.737034    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-token\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737111 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.737090    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979c607b-69fa-4ac6-ab3e-03c7550e4274-istiod-ca-cert\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737322 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.737120    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-credential-socket\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737322 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.737140    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-socket\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737322 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.737164    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-podinfo\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737322 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.737188    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49mkc\" (UniqueName: \"kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-kube-api-access-49mkc\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.737515    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-certs\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737633 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.737554    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-data\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737704 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.737635    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-credential-socket\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.737998 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.737981    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979c607b-69fa-4ac6-ab3e-03c7550e4274-istiod-ca-cert\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.738047 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.738018    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-socket\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.739369 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.739340    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-envoy\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.739740 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.739718    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-podinfo\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.751215 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.751189    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-token\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.751534 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.751513    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49mkc\" (UniqueName: \"kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-kube-api-access-49mkc\") pod \"e2e-mt-699a3596-openshift-default-648f9595d7-4gzht\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") " pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.845339 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.845243    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:30.994447 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.994403    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"]
Jul 15 04:40:30.996957 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:40:30.996926    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979c607b_69fa_4ac6_ab3e_03c7550e4274.slice/crio-cb7a1b40eae70fa271d061675eecd09b920b35b1e839d5389ab81bc050f2eaa3 WatchSource:0}: Error finding container cb7a1b40eae70fa271d061675eecd09b920b35b1e839d5389ab81bc050f2eaa3: Status 404 returned error can't find the container with id cb7a1b40eae70fa271d061675eecd09b920b35b1e839d5389ab81bc050f2eaa3
Jul 15 04:40:30.999192 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.999157    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:40:30.999273 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.999230    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:40:30.999273 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:30.999256    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:40:31.942986 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:31.942946    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" event={"ID":"979c607b-69fa-4ac6-ab3e-03c7550e4274","Type":"ContainerStarted","Data":"d92f5b53fc82290b7c20c511ffce130dfb14ed8967e01312616f511cf71c99d3"}
Jul 15 04:40:31.943406 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:31.943012    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" event={"ID":"979c607b-69fa-4ac6-ab3e-03c7550e4274","Type":"ContainerStarted","Data":"cb7a1b40eae70fa271d061675eecd09b920b35b1e839d5389ab81bc050f2eaa3"}
Jul 15 04:40:31.974658 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:31.974584    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" podStartSLOduration=1.9745674800000002 podStartE2EDuration="1.97456748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:40:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:40:31.972560672 +0000 UTC m=+2061.283941913" watchObservedRunningTime="2026-07-15 04:40:31.97456748 +0000 UTC m=+2061.285948726"
Jul 15 04:40:32.845750 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:32.845665    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:32.850562 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:32.850540    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:32.947535 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:32.947497    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:32.948419 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:32.948399    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:34.122775 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.122735    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"]
Jul 15 04:40:34.126238 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.126212    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:34.130248 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.130226    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-699a3596-serving-cert\""
Jul 15 04:40:34.150490 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.150456    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"]
Jul 15 04:40:34.272655 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.272578    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fae538fc-0549-4586-a703-6915dbec871a-maas-api-tls\") pod \"maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:34.272838 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.272760    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae538fc-0549-4586-a703-6915dbec871a-openshift-service-ca\") pod \"maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:34.272838 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.272822    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szjd5\" (UniqueName: \"kubernetes.io/projected/fae538fc-0549-4586-a703-6915dbec871a-kube-api-access-szjd5\") pod \"maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:34.373689 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.373556    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae538fc-0549-4586-a703-6915dbec871a-openshift-service-ca\") pod \"maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:34.373689 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.373640    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjd5\" (UniqueName: \"kubernetes.io/projected/fae538fc-0549-4586-a703-6915dbec871a-kube-api-access-szjd5\") pod \"maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:34.373689 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.373685    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fae538fc-0549-4586-a703-6915dbec871a-maas-api-tls\") pod \"maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:34.373953 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:40:34.373810    2524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-serving-cert: secret "maas-api-e2e-mt-699a3596-serving-cert" not found
Jul 15 04:40:34.373953 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:40:34.373874    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae538fc-0549-4586-a703-6915dbec871a-maas-api-tls podName:fae538fc-0549-4586-a703-6915dbec871a nodeName:}" failed. No retries permitted until 2026-07-15 04:40:34.873856635 +0000 UTC m=+2064.185237860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fae538fc-0549-4586-a703-6915dbec871a-maas-api-tls") pod "maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t" (UID: "fae538fc-0549-4586-a703-6915dbec871a") : secret "maas-api-e2e-mt-699a3596-serving-cert" not found
Jul 15 04:40:34.374498 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.374471    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae538fc-0549-4586-a703-6915dbec871a-openshift-service-ca\") pod \"maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:34.383027 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.382999    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szjd5\" (UniqueName: \"kubernetes.io/projected/fae538fc-0549-4586-a703-6915dbec871a-kube-api-access-szjd5\") pod \"maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:34.878514 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.878478    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fae538fc-0549-4586-a703-6915dbec871a-maas-api-tls\") pod \"maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:34.881255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:34.881224    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fae538fc-0549-4586-a703-6915dbec871a-maas-api-tls\") pod \"maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:35.040073 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:35.040038    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:35.181745 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:35.181719    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"]
Jul 15 04:40:35.185347 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:40:35.185315    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae538fc_0549_4586_a703_6915dbec871a.slice/crio-21c8ffb2b288b4e960188f30cd20379026a108a6ef75ee0ad208706186747f13 WatchSource:0}: Error finding container 21c8ffb2b288b4e960188f30cd20379026a108a6ef75ee0ad208706186747f13: Status 404 returned error can't find the container with id 21c8ffb2b288b4e960188f30cd20379026a108a6ef75ee0ad208706186747f13
Jul 15 04:40:35.962470 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:35.962427    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t" event={"ID":"fae538fc-0549-4586-a703-6915dbec871a","Type":"ContainerStarted","Data":"97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27"}
Jul 15 04:40:35.962723 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:35.962483    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t" event={"ID":"fae538fc-0549-4586-a703-6915dbec871a","Type":"ContainerStarted","Data":"21c8ffb2b288b4e960188f30cd20379026a108a6ef75ee0ad208706186747f13"}
Jul 15 04:40:35.962723 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:35.962504    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:35.980324 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:35.980271    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t" podStartSLOduration=1.552832141 podStartE2EDuration="1.980255085s" totalImagesPullingTime="427.422944ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:40:35.978799641 +0000 UTC m=+2065.290180878" watchObservedRunningTime="2026-07-15 04:40:35.980255085 +0000 UTC m=+2065.291636332"
Jul 15 04:40:41.982984 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:41.982946    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:40:47.020309 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:47.020218    2524 generic.go:356] "Generic (PLEG): container finished" podID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerID="d92f5b53fc82290b7c20c511ffce130dfb14ed8967e01312616f511cf71c99d3" exitCode=0
Jul 15 04:40:47.020309 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:47.020293    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" event={"ID":"979c607b-69fa-4ac6-ab3e-03c7550e4274","Type":"ContainerDied","Data":"d92f5b53fc82290b7c20c511ffce130dfb14ed8967e01312616f511cf71c99d3"}
Jul 15 04:40:47.020800 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:47.020737    2524 scope.go:117] "RemoveContainer" containerID="d92f5b53fc82290b7c20c511ffce130dfb14ed8967e01312616f511cf71c99d3"
Jul 15 04:40:47.021804 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:47.021766    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:40:47.021937 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:47.021839    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:40:47.021937 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:47.021880    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:40:47.948498 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:47.948452    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:48.026678 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:48.026641    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" event={"ID":"979c607b-69fa-4ac6-ab3e-03c7550e4274","Type":"ContainerStarted","Data":"18f1b9b6ec04032d574c4f5a36ab13de26cf95fd4380c5730b3f26438945ad9a"}
Jul 15 04:40:48.845689 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:48.845645    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:48.847065 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:48.847037    2524 patch_prober.go:28] interesting pod/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 15 04:40:48.847193 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:48.847087    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 15 04:40:49.846172 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:49.846133    2524 patch_prober.go:28] interesting pod/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 15 04:40:49.846572 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:49.846207    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 15 04:40:50.845810 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:50.845768    2524 patch_prober.go:28] interesting pod/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 15 04:40:50.846007 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:50.845849    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 15 04:40:51.846270 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:51.846232    2524 patch_prober.go:28] interesting pod/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 15 04:40:51.846724 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:51.846303    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 15 04:40:52.850610 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:52.850566    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:52.851081 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:52.851067    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:40:52.851880 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:40:52.851858    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:41:11.450159 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:11.450128    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:41:11.452886 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:11.452862    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:41:17.312060 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.312027    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"]
Jul 15 04:41:17.312570 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.312226    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t" podUID="fae538fc-0549-4586-a703-6915dbec871a" containerName="maas-api" containerID="cri-o://97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27" gracePeriod=30
Jul 15 04:41:17.566821 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.566757    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:41:17.671431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.671391    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae538fc-0549-4586-a703-6915dbec871a-openshift-service-ca\") pod \"fae538fc-0549-4586-a703-6915dbec871a\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") "
Jul 15 04:41:17.671431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.671431    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szjd5\" (UniqueName: \"kubernetes.io/projected/fae538fc-0549-4586-a703-6915dbec871a-kube-api-access-szjd5\") pod \"fae538fc-0549-4586-a703-6915dbec871a\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") "
Jul 15 04:41:17.671725 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.671477    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fae538fc-0549-4586-a703-6915dbec871a-maas-api-tls\") pod \"fae538fc-0549-4586-a703-6915dbec871a\" (UID: \"fae538fc-0549-4586-a703-6915dbec871a\") "
Jul 15 04:41:17.671852 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.671831    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fae538fc-0549-4586-a703-6915dbec871a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fae538fc-0549-4586-a703-6915dbec871a" (UID: "fae538fc-0549-4586-a703-6915dbec871a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:41:17.673809 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.673781    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fae538fc-0549-4586-a703-6915dbec871a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fae538fc-0549-4586-a703-6915dbec871a" (UID: "fae538fc-0549-4586-a703-6915dbec871a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:41:17.673917 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.673889    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fae538fc-0549-4586-a703-6915dbec871a-kube-api-access-szjd5" (OuterVolumeSpecName: "kube-api-access-szjd5") pod "fae538fc-0549-4586-a703-6915dbec871a" (UID: "fae538fc-0549-4586-a703-6915dbec871a"). InnerVolumeSpecName "kube-api-access-szjd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:41:17.772371 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.772332    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae538fc-0549-4586-a703-6915dbec871a-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:41:17.772371 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.772363    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szjd5\" (UniqueName: \"kubernetes.io/projected/fae538fc-0549-4586-a703-6915dbec871a-kube-api-access-szjd5\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:41:17.772371 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:17.772374    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fae538fc-0549-4586-a703-6915dbec871a-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:41:18.166381 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:18.166347    2524 generic.go:356] "Generic (PLEG): container finished" podID="fae538fc-0549-4586-a703-6915dbec871a" containerID="97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27" exitCode=0
Jul 15 04:41:18.166567 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:18.166395    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"
Jul 15 04:41:18.166567 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:18.166443    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t" event={"ID":"fae538fc-0549-4586-a703-6915dbec871a","Type":"ContainerDied","Data":"97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27"}
Jul 15 04:41:18.166567 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:18.166504    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t" event={"ID":"fae538fc-0549-4586-a703-6915dbec871a","Type":"ContainerDied","Data":"21c8ffb2b288b4e960188f30cd20379026a108a6ef75ee0ad208706186747f13"}
Jul 15 04:41:18.166567 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:18.166531    2524 scope.go:117] "RemoveContainer" containerID="97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27"
Jul 15 04:41:18.175189 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:18.175173    2524 scope.go:117] "RemoveContainer" containerID="97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27"
Jul 15 04:41:18.175450 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:41:18.175432    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27\": container with ID starting with 97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27 not found: ID does not exist" containerID="97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27"
Jul 15 04:41:18.175498 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:18.175458    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27"} err="failed to get container status \"97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27\": rpc error: code = NotFound desc = could not find container \"97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27\": container with ID starting with 97155b783ca2f7d6cddaf0e86ebe6e10130e05ee0fb838a6682e75abe825af27 not found: ID does not exist"
Jul 15 04:41:18.188195 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:18.188165    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"]
Jul 15 04:41:18.192079 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:18.192060    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-699a3596-76b66c9c9b-k6g9t"]
Jul 15 04:41:19.432189 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:41:19.432155    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fae538fc-0549-4586-a703-6915dbec871a" path="/var/lib/kubelet/pods/fae538fc-0549-4586-a703-6915dbec871a/volumes"
Jul 15 04:42:14.319160 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:14.319123    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"]
Jul 15 04:42:14.319723 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:14.319375    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerName="istio-proxy" containerID="cri-o://18f1b9b6ec04032d574c4f5a36ab13de26cf95fd4380c5730b3f26438945ad9a" gracePeriod=30
Jul 15 04:42:19.474008 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.473971    2524 generic.go:356] "Generic (PLEG): container finished" podID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerID="18f1b9b6ec04032d574c4f5a36ab13de26cf95fd4380c5730b3f26438945ad9a" exitCode=0
Jul 15 04:42:19.474452 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.474036    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" event={"ID":"979c607b-69fa-4ac6-ab3e-03c7550e4274","Type":"ContainerDied","Data":"18f1b9b6ec04032d574c4f5a36ab13de26cf95fd4380c5730b3f26438945ad9a"}
Jul 15 04:42:19.474452 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.474089    2524 scope.go:117] "RemoveContainer" containerID="d92f5b53fc82290b7c20c511ffce130dfb14ed8967e01312616f511cf71c99d3"
Jul 15 04:42:19.588482 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.588389    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:42:19.633625 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.633571    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979c607b-69fa-4ac6-ab3e-03c7550e4274-istiod-ca-cert\") pod \"979c607b-69fa-4ac6-ab3e-03c7550e4274\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") "
Jul 15 04:42:19.633853 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.633664    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-certs\") pod \"979c607b-69fa-4ac6-ab3e-03c7550e4274\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") "
Jul 15 04:42:19.633853 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.633684    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-token\") pod \"979c607b-69fa-4ac6-ab3e-03c7550e4274\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") "
Jul 15 04:42:19.633853 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.633712    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-envoy\") pod \"979c607b-69fa-4ac6-ab3e-03c7550e4274\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") "
Jul 15 04:42:19.633853 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.633736    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-podinfo\") pod \"979c607b-69fa-4ac6-ab3e-03c7550e4274\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") "
Jul 15 04:42:19.633853 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.633797    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-data\") pod \"979c607b-69fa-4ac6-ab3e-03c7550e4274\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") "
Jul 15 04:42:19.633853 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.633842    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-credential-socket\") pod \"979c607b-69fa-4ac6-ab3e-03c7550e4274\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") "
Jul 15 04:42:19.634184 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.633871    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-socket\") pod \"979c607b-69fa-4ac6-ab3e-03c7550e4274\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") "
Jul 15 04:42:19.634184 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.633896    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49mkc\" (UniqueName: \"kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-kube-api-access-49mkc\") pod \"979c607b-69fa-4ac6-ab3e-03c7550e4274\" (UID: \"979c607b-69fa-4ac6-ab3e-03c7550e4274\") "
Jul 15 04:42:19.634184 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.634044    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "979c607b-69fa-4ac6-ab3e-03c7550e4274" (UID: "979c607b-69fa-4ac6-ab3e-03c7550e4274"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:42:19.634353 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.634291    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:42:19.634353 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.634292    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "979c607b-69fa-4ac6-ab3e-03c7550e4274" (UID: "979c607b-69fa-4ac6-ab3e-03c7550e4274"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:42:19.634459 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.634348    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/979c607b-69fa-4ac6-ab3e-03c7550e4274-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "979c607b-69fa-4ac6-ab3e-03c7550e4274" (UID: "979c607b-69fa-4ac6-ab3e-03c7550e4274"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:42:19.634459 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.634361    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "979c607b-69fa-4ac6-ab3e-03c7550e4274" (UID: "979c607b-69fa-4ac6-ab3e-03c7550e4274"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:42:19.634876 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.634838    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-data" (OuterVolumeSpecName: "istio-data") pod "979c607b-69fa-4ac6-ab3e-03c7550e4274" (UID: "979c607b-69fa-4ac6-ab3e-03c7550e4274"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:42:19.636449 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.636417    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-kube-api-access-49mkc" (OuterVolumeSpecName: "kube-api-access-49mkc") pod "979c607b-69fa-4ac6-ab3e-03c7550e4274" (UID: "979c607b-69fa-4ac6-ab3e-03c7550e4274"). InnerVolumeSpecName "kube-api-access-49mkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:42:19.636814 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.636792    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "979c607b-69fa-4ac6-ab3e-03c7550e4274" (UID: "979c607b-69fa-4ac6-ab3e-03c7550e4274"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:42:19.636814 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.636802    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-token" (OuterVolumeSpecName: "istio-token") pod "979c607b-69fa-4ac6-ab3e-03c7550e4274" (UID: "979c607b-69fa-4ac6-ab3e-03c7550e4274"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:42:19.636951 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.636827    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "979c607b-69fa-4ac6-ab3e-03c7550e4274" (UID: "979c607b-69fa-4ac6-ab3e-03c7550e4274"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:42:19.735455 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.735405    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979c607b-69fa-4ac6-ab3e-03c7550e4274-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:42:19.735455 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.735441    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:42:19.735455 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.735449    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:42:19.735455 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.735457    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:42:19.735455 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.735465    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:42:19.735455 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.735474    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:42:19.735879 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.735484    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c607b-69fa-4ac6-ab3e-03c7550e4274-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:42:19.735879 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:19.735493    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49mkc\" (UniqueName: \"kubernetes.io/projected/979c607b-69fa-4ac6-ab3e-03c7550e4274-kube-api-access-49mkc\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:42:20.480378 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:20.480338    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"
Jul 15 04:42:20.480871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:20.480378    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht" event={"ID":"979c607b-69fa-4ac6-ab3e-03c7550e4274","Type":"ContainerDied","Data":"cb7a1b40eae70fa271d061675eecd09b920b35b1e839d5389ab81bc050f2eaa3"}
Jul 15 04:42:20.480871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:20.480439    2524 scope.go:117] "RemoveContainer" containerID="18f1b9b6ec04032d574c4f5a36ab13de26cf95fd4380c5730b3f26438945ad9a"
Jul 15 04:42:20.504072 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:20.504035    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"]
Jul 15 04:42:20.505889 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:20.505859    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-699a3596-openshift-default-648f9595d7-4gzht"]
Jul 15 04:42:21.433331 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:42:21.433298    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" path="/var/lib/kubelet/pods/979c607b-69fa-4ac6-ab3e-03c7550e4274/volumes"
Jul 15 04:43:09.162422 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.162355    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"]
Jul 15 04:43:09.163011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.162931    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fae538fc-0549-4586-a703-6915dbec871a" containerName="maas-api"
Jul 15 04:43:09.163011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.162956    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="fae538fc-0549-4586-a703-6915dbec871a" containerName="maas-api"
Jul 15 04:43:09.163011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.162975    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerName="istio-proxy"
Jul 15 04:43:09.163011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.162980    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerName="istio-proxy"
Jul 15 04:43:09.163011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.162993    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerName="istio-proxy"
Jul 15 04:43:09.163011 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.162999    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerName="istio-proxy"
Jul 15 04:43:09.163295 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.163084    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerName="istio-proxy"
Jul 15 04:43:09.163295 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.163102    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="fae538fc-0549-4586-a703-6915dbec871a" containerName="maas-api"
Jul 15 04:43:09.163295 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.163112    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="979c607b-69fa-4ac6-ab3e-03c7550e4274" containerName="istio-proxy"
Jul 15 04:43:09.166368 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.166348    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.169026 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.168992    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-005cfa-openshift-default-dockercfg-vzhhh\""
Jul 15 04:43:09.180036 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.179997    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"]
Jul 15 04:43:09.282529 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.282483    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-envoy\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.282529 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.282526    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-podinfo\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.282830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.282566    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6wxg\" (UniqueName: \"kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-kube-api-access-t6wxg\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.282830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.282633    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-certs\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.282830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.282679    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-credential-socket\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.282830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.282711    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-socket\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.282830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.282732    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-data\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.282830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.282803    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istiod-ca-cert\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.283054 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.282886    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-token\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384142 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384100    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-certs\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384142 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384147    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-credential-socket\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384386 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384166    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-socket\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384386 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384277    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-data\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384386 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384322    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istiod-ca-cert\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384533 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384404    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-token\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384533 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384444    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-envoy\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384533 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384469    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-podinfo\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384710 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384540    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6wxg\" (UniqueName: \"kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-kube-api-access-t6wxg\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384710 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384543    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-credential-socket\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384793 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384721    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-certs\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384872 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384853    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-socket\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.384984 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.384958    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-data\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.385134 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.385110    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istiod-ca-cert\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.387013 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.386994    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-envoy\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.387294 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.387274    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-podinfo\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.392531 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.392500    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-token\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.392678 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.392643    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6wxg\" (UniqueName: \"kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-kube-api-access-t6wxg\") pod \"e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") " pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.477038 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.476930    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:09.625835 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.625798    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"]
Jul 15 04:43:09.627628 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:43:09.627581    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9452a6e6_11d2_49b8_9f1f_54ede62c6ce6.slice/crio-c72f857700bc094bd91c554f87572b2fe31af33143b1e5454d9586a8cd43f24b WatchSource:0}: Error finding container c72f857700bc094bd91c554f87572b2fe31af33143b1e5454d9586a8cd43f24b: Status 404 returned error can't find the container with id c72f857700bc094bd91c554f87572b2fe31af33143b1e5454d9586a8cd43f24b
Jul 15 04:43:09.630172 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.630138    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:09.630271 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.630208    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:09.630271 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.630238    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:09.724087 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.724045    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" event={"ID":"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6","Type":"ContainerStarted","Data":"69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287"}
Jul 15 04:43:09.724270 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.724117    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" event={"ID":"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6","Type":"ContainerStarted","Data":"c72f857700bc094bd91c554f87572b2fe31af33143b1e5454d9586a8cd43f24b"}
Jul 15 04:43:09.744926 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:09.744791    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" podStartSLOduration=0.744766645 podStartE2EDuration="744.766645ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:43:09.742835499 +0000 UTC m=+2219.054216773" watchObservedRunningTime="2026-07-15 04:43:09.744766645 +0000 UTC m=+2219.056147892"
Jul 15 04:43:10.477844 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:10.477806    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:10.483153 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:10.483125    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:10.729172 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:10.729076    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:10.730161 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:10.730140    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:11.910437 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:11.909876    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"]
Jul 15 04:43:11.915977 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:11.915941    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:11.918385 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:11.918353    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-005cfa-serving-cert\""
Jul 15 04:43:11.926797 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:11.926431    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"]
Jul 15 04:43:12.010024 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.009976    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztgrd\" (UniqueName: \"kubernetes.io/projected/63db4cd3-59ce-4c36-9968-d62b7e7165b8-kube-api-access-ztgrd\") pod \"maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:12.010024 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.010023    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63db4cd3-59ce-4c36-9968-d62b7e7165b8-maas-api-tls\") pod \"maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:12.010315 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.010167    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63db4cd3-59ce-4c36-9968-d62b7e7165b8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:12.111703 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.111647    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63db4cd3-59ce-4c36-9968-d62b7e7165b8-maas-api-tls\") pod \"maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:12.111931 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.111774    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63db4cd3-59ce-4c36-9968-d62b7e7165b8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:12.111931 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:43:12.111811    2524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-serving-cert: secret "maas-api-e2e-inf-a-005cfa-serving-cert" not found
Jul 15 04:43:12.111931 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.111860    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztgrd\" (UniqueName: \"kubernetes.io/projected/63db4cd3-59ce-4c36-9968-d62b7e7165b8-kube-api-access-ztgrd\") pod \"maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:12.111931 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:43:12.111896    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63db4cd3-59ce-4c36-9968-d62b7e7165b8-maas-api-tls podName:63db4cd3-59ce-4c36-9968-d62b7e7165b8 nodeName:}" failed. No retries permitted until 2026-07-15 04:43:12.611879068 +0000 UTC m=+2221.923260296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63db4cd3-59ce-4c36-9968-d62b7e7165b8-maas-api-tls") pod "maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt" (UID: "63db4cd3-59ce-4c36-9968-d62b7e7165b8") : secret "maas-api-e2e-inf-a-005cfa-serving-cert" not found
Jul 15 04:43:12.112476 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.112454    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63db4cd3-59ce-4c36-9968-d62b7e7165b8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:12.120733 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.120704    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztgrd\" (UniqueName: \"kubernetes.io/projected/63db4cd3-59ce-4c36-9968-d62b7e7165b8-kube-api-access-ztgrd\") pod \"maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:12.616194 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.616151    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63db4cd3-59ce-4c36-9968-d62b7e7165b8-maas-api-tls\") pod \"maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:12.618915 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.618891    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63db4cd3-59ce-4c36-9968-d62b7e7165b8-maas-api-tls\") pod \"maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:12.832372 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.832332    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:12.975913 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.975812    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"]
Jul 15 04:43:12.978267 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:43:12.978238    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63db4cd3_59ce_4c36_9968_d62b7e7165b8.slice/crio-bc60da947876881b394da8914cb7669e6d0b4cc70d1cb19c4eb265a1d0847363 WatchSource:0}: Error finding container bc60da947876881b394da8914cb7669e6d0b4cc70d1cb19c4eb265a1d0847363: Status 404 returned error can't find the container with id bc60da947876881b394da8914cb7669e6d0b4cc70d1cb19c4eb265a1d0847363
Jul 15 04:43:12.979806 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:12.979787    2524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 04:43:13.746198 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:13.746093    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt" event={"ID":"63db4cd3-59ce-4c36-9968-d62b7e7165b8","Type":"ContainerStarted","Data":"27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31"}
Jul 15 04:43:13.746198 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:13.746161    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:13.746198 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:13.746177    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt" event={"ID":"63db4cd3-59ce-4c36-9968-d62b7e7165b8","Type":"ContainerStarted","Data":"bc60da947876881b394da8914cb7669e6d0b4cc70d1cb19c4eb265a1d0847363"}
Jul 15 04:43:13.765427 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:13.765368    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt" podStartSLOduration=2.373109597 podStartE2EDuration="2.765351363s" totalImagesPullingTime="392.241766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:43:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:43:13.763288513 +0000 UTC m=+2223.074669759" watchObservedRunningTime="2026-07-15 04:43:13.765351363 +0000 UTC m=+2223.076732609"
Jul 15 04:43:16.862970 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.862931    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"]
Jul 15 04:43:16.867424 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.867392    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:16.871901 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.870341    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-194ff1-openshift-default-dockercfg-zhwh9\""
Jul 15 04:43:16.879128 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.879097    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"]
Jul 15 04:43:16.956120 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.956073    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istiod-ca-cert\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:16.956333 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.956139    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x759d\" (UniqueName: \"kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-kube-api-access-x759d\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:16.956333 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.956195    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-socket\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:16.956333 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.956247    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-token\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:16.956523 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.956325    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-credential-socket\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:16.956523 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.956367    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-certs\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:16.956523 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.956393    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-envoy\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:16.956523 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.956465    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-podinfo\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:16.956782 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:16.956526    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-data\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058027 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.057984    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istiod-ca-cert\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058027 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058024    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x759d\" (UniqueName: \"kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-kube-api-access-x759d\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058296 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058058    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-socket\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058296 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058190    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-token\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058296 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058243    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-credential-socket\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058296 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058285    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-certs\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058309    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-envoy\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058371    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-podinfo\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058423    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-data\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058484    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-socket\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058744 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058584    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-credential-socket\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058744 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058668    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-certs\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058858 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058791    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istiod-ca-cert\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.058858 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.058801    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-data\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.060975 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.060951    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-envoy\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.061179 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.061162    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-podinfo\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.066417 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.066386    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x759d\" (UniqueName: \"kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-kube-api-access-x759d\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.066523 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.066497    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-token\") pod \"e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") " pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.186174 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.186132    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:17.339311 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.339285    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"]
Jul 15 04:43:17.341674 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:43:17.341634    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac3931d_6a31_4b84_862a_07d8e1f4020c.slice/crio-eed0c9850a4613b80e4495d18986de95a86b765dc7ca604d69268fd9ebd152d9 WatchSource:0}: Error finding container eed0c9850a4613b80e4495d18986de95a86b765dc7ca604d69268fd9ebd152d9: Status 404 returned error can't find the container with id eed0c9850a4613b80e4495d18986de95a86b765dc7ca604d69268fd9ebd152d9
Jul 15 04:43:17.350725 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.345211    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:17.350725 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.345371    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:17.350725 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.345455    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:17.770586 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.770533    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" event={"ID":"1ac3931d-6a31-4b84-862a-07d8e1f4020c","Type":"ContainerStarted","Data":"d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097"}
Jul 15 04:43:17.770815 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.770631    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" event={"ID":"1ac3931d-6a31-4b84-862a-07d8e1f4020c","Type":"ContainerStarted","Data":"eed0c9850a4613b80e4495d18986de95a86b765dc7ca604d69268fd9ebd152d9"}
Jul 15 04:43:17.790632 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:17.790552    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" podStartSLOduration=1.7905332459999999 podStartE2EDuration="1.790533246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:43:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:43:17.788670005 +0000 UTC m=+2227.100051253" watchObservedRunningTime="2026-07-15 04:43:17.790533246 +0000 UTC m=+2227.101914493"
Jul 15 04:43:18.187342 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:18.187293    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:18.193496 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:18.193467    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:18.779337 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:18.779296    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:18.784283 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:18.784254    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:19.756987 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:19.756957    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:43:19.945208 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:19.945174    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"]
Jul 15 04:43:19.948814 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:19.948790    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:19.951789 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:19.951761    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-194ff1-serving-cert\""
Jul 15 04:43:19.971321 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:19.971287    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"]
Jul 15 04:43:20.094556 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.094457    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rld7z\" (UniqueName: \"kubernetes.io/projected/3326dc69-29e0-4029-b619-b6c722d0b93b-kube-api-access-rld7z\") pod \"maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:20.094556 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.094506    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3326dc69-29e0-4029-b619-b6c722d0b93b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:20.094785 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.094652    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3326dc69-29e0-4029-b619-b6c722d0b93b-maas-api-tls\") pod \"maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:20.195203 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.195164    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rld7z\" (UniqueName: \"kubernetes.io/projected/3326dc69-29e0-4029-b619-b6c722d0b93b-kube-api-access-rld7z\") pod \"maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:20.195203 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.195216    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3326dc69-29e0-4029-b619-b6c722d0b93b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:20.195481 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.195290    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3326dc69-29e0-4029-b619-b6c722d0b93b-maas-api-tls\") pod \"maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:20.195920 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.195899    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3326dc69-29e0-4029-b619-b6c722d0b93b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:20.197974 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.197952    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3326dc69-29e0-4029-b619-b6c722d0b93b-maas-api-tls\") pod \"maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:20.204387 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.204353    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rld7z\" (UniqueName: \"kubernetes.io/projected/3326dc69-29e0-4029-b619-b6c722d0b93b-kube-api-access-rld7z\") pod \"maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:20.272724 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.272674    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:20.446404 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.446373    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"]
Jul 15 04:43:20.448530 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:43:20.448501    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3326dc69_29e0_4029_b619_b6c722d0b93b.slice/crio-9b9e6c3c270de8af74e2f809f3746ed36b46a1f554a5827cc043d36e254bd539 WatchSource:0}: Error finding container 9b9e6c3c270de8af74e2f809f3746ed36b46a1f554a5827cc043d36e254bd539: Status 404 returned error can't find the container with id 9b9e6c3c270de8af74e2f809f3746ed36b46a1f554a5827cc043d36e254bd539
Jul 15 04:43:20.794871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:20.794836    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7" event={"ID":"3326dc69-29e0-4029-b619-b6c722d0b93b","Type":"ContainerStarted","Data":"9b9e6c3c270de8af74e2f809f3746ed36b46a1f554a5827cc043d36e254bd539"}
Jul 15 04:43:21.802690 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:21.802638    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7" event={"ID":"3326dc69-29e0-4029-b619-b6c722d0b93b","Type":"ContainerStarted","Data":"5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1"}
Jul 15 04:43:21.803158 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:21.802778    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:21.819763 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:21.819702    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7" podStartSLOduration=2.500262858 podStartE2EDuration="2.819684247s" totalImagesPullingTime="319.421389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:43:21.818962279 +0000 UTC m=+2231.130343527" watchObservedRunningTime="2026-07-15 04:43:21.819684247 +0000 UTC m=+2231.131065494"
Jul 15 04:43:24.819438 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:24.819358    2524 generic.go:356] "Generic (PLEG): container finished" podID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerID="69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287" exitCode=0
Jul 15 04:43:24.819438 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:24.819426    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" event={"ID":"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6","Type":"ContainerDied","Data":"69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287"}
Jul 15 04:43:24.819958 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:24.819871    2524 scope.go:117] "RemoveContainer" containerID="69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287"
Jul 15 04:43:24.820997 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:24.820968    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:24.821081 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:24.821037    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:24.821081 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:24.821069    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:25.729539 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:25.729490    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:25.826303 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:25.826272    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" event={"ID":"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6","Type":"ContainerStarted","Data":"faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d"}
Jul 15 04:43:26.479342 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:26.478865    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:26.479342 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:26.479008    2524 patch_prober.go:28] interesting pod/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 15 04:43:26.479342 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:26.479056    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 15 04:43:27.478376 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:27.478326    2524 patch_prober.go:28] interesting pod/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 15 04:43:27.478937 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:27.478415    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 15 04:43:27.814973 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:27.814880    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:43:28.478339 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:28.478295    2524 patch_prober.go:28] interesting pod/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 15 04:43:28.478718 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:28.478682    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 15 04:43:29.478579 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:29.478526    2524 patch_prober.go:28] interesting pod/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 15 04:43:29.478830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:29.478652    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 15 04:43:30.483376 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:30.483340    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:30.483874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:30.483770    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:30.484637 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:30.484612    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:43:33.780423 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:33.780382    2524 patch_prober.go:28] interesting pod/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 15 04:43:33.780899 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:33.780450    2524 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 15 04:43:33.872933 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:33.872897    2524 generic.go:356] "Generic (PLEG): container finished" podID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerID="d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097" exitCode=0
Jul 15 04:43:33.873109 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:33.872981    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" event={"ID":"1ac3931d-6a31-4b84-862a-07d8e1f4020c","Type":"ContainerDied","Data":"d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097"}
Jul 15 04:43:33.873430 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:33.873410    2524 scope.go:117] "RemoveContainer" containerID="d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097"
Jul 15 04:43:33.874617 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:33.874560    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:33.874692 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:33.874680    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:33.874730 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:33.874719    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:43:34.884152 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:34.884118    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" event={"ID":"1ac3931d-6a31-4b84-862a-07d8e1f4020c","Type":"ContainerStarted","Data":"4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80"}
Jul 15 04:43:35.186706 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:35.186519    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:35.186913 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:35.186887    2524 patch_prober.go:28] interesting pod/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 15 04:43:35.186996 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:35.186945    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 15 04:43:36.187300 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:36.187264    2524 patch_prober.go:28] interesting pod/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 15 04:43:36.187759 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:36.187323    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 15 04:43:37.187232 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:37.187187    2524 patch_prober.go:28] interesting pod/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 15 04:43:37.187430 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:37.187254    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 15 04:43:38.186837 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:38.186789    2524 patch_prober.go:28] interesting pod/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 15 04:43:38.187043 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:38.186882    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 15 04:43:39.190498 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:39.190462    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:39.191017 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:39.190875    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:43:39.191771 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:43:39.191751    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:44:34.400448 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.400407    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"]
Jul 15 04:44:34.401140 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.400681    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt" podUID="63db4cd3-59ce-4c36-9968-d62b7e7165b8" containerName="maas-api" containerID="cri-o://27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31" gracePeriod=30
Jul 15 04:44:34.669445 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.669363    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:44:34.751230 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.751192    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63db4cd3-59ce-4c36-9968-d62b7e7165b8-openshift-service-ca\") pod \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") "
Jul 15 04:44:34.751230 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.751248    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63db4cd3-59ce-4c36-9968-d62b7e7165b8-maas-api-tls\") pod \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") "
Jul 15 04:44:34.751488 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.751307    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztgrd\" (UniqueName: \"kubernetes.io/projected/63db4cd3-59ce-4c36-9968-d62b7e7165b8-kube-api-access-ztgrd\") pod \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\" (UID: \"63db4cd3-59ce-4c36-9968-d62b7e7165b8\") "
Jul 15 04:44:34.751622 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.751569    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63db4cd3-59ce-4c36-9968-d62b7e7165b8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63db4cd3-59ce-4c36-9968-d62b7e7165b8" (UID: "63db4cd3-59ce-4c36-9968-d62b7e7165b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:44:34.753585 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.753552    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63db4cd3-59ce-4c36-9968-d62b7e7165b8-kube-api-access-ztgrd" (OuterVolumeSpecName: "kube-api-access-ztgrd") pod "63db4cd3-59ce-4c36-9968-d62b7e7165b8" (UID: "63db4cd3-59ce-4c36-9968-d62b7e7165b8"). InnerVolumeSpecName "kube-api-access-ztgrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:44:34.753731 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.753664    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63db4cd3-59ce-4c36-9968-d62b7e7165b8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63db4cd3-59ce-4c36-9968-d62b7e7165b8" (UID: "63db4cd3-59ce-4c36-9968-d62b7e7165b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:44:34.852764 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.852721    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63db4cd3-59ce-4c36-9968-d62b7e7165b8-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:34.852764 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.852757    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63db4cd3-59ce-4c36-9968-d62b7e7165b8-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:34.852764 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:34.852767    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztgrd\" (UniqueName: \"kubernetes.io/projected/63db4cd3-59ce-4c36-9968-d62b7e7165b8-kube-api-access-ztgrd\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:35.184516 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:35.184482    2524 generic.go:356] "Generic (PLEG): container finished" podID="63db4cd3-59ce-4c36-9968-d62b7e7165b8" containerID="27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31" exitCode=0
Jul 15 04:44:35.184727 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:35.184555    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"
Jul 15 04:44:35.184727 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:35.184569    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt" event={"ID":"63db4cd3-59ce-4c36-9968-d62b7e7165b8","Type":"ContainerDied","Data":"27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31"}
Jul 15 04:44:35.184727 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:35.184653    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt" event={"ID":"63db4cd3-59ce-4c36-9968-d62b7e7165b8","Type":"ContainerDied","Data":"bc60da947876881b394da8914cb7669e6d0b4cc70d1cb19c4eb265a1d0847363"}
Jul 15 04:44:35.184727 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:35.184679    2524 scope.go:117] "RemoveContainer" containerID="27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31"
Jul 15 04:44:35.194972 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:35.194953    2524 scope.go:117] "RemoveContainer" containerID="27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31"
Jul 15 04:44:35.195541 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:44:35.195437    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31\": container with ID starting with 27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31 not found: ID does not exist" containerID="27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31"
Jul 15 04:44:35.195541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:35.195476    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31"} err="failed to get container status \"27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31\": rpc error: code = NotFound desc = could not find container \"27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31\": container with ID starting with 27f4b3cbd06612b41a45c51639fd7b82a548876cc5c7d9ae782ab53a3d1eac31 not found: ID does not exist"
Jul 15 04:44:35.209501 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:35.209468    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"]
Jul 15 04:44:35.215118 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:35.215082    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-005cfa-8767f9db9-76dxt"]
Jul 15 04:44:35.432915 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:35.432876    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63db4cd3-59ce-4c36-9968-d62b7e7165b8" path="/var/lib/kubelet/pods/63db4cd3-59ce-4c36-9968-d62b7e7165b8/volumes"
Jul 15 04:44:49.768459 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:49.768417    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"]
Jul 15 04:44:49.768973 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:49.768714    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerName="istio-proxy" containerID="cri-o://faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d" gracePeriod=30
Jul 15 04:44:55.038309 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.038234    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:44:55.055015 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.054981    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-data\") pod \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") "
Jul 15 04:44:55.055196 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055033    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istiod-ca-cert\") pod \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") "
Jul 15 04:44:55.055196 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055061    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-certs\") pod \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") "
Jul 15 04:44:55.055196 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055079    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-envoy\") pod \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") "
Jul 15 04:44:55.055196 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055111    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-podinfo\") pod \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") "
Jul 15 04:44:55.055196 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055134    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6wxg\" (UniqueName: \"kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-kube-api-access-t6wxg\") pod \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") "
Jul 15 04:44:55.055196 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055155    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-credential-socket\") pod \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") "
Jul 15 04:44:55.055196 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055175    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-token\") pod \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") "
Jul 15 04:44:55.055575 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055452    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" (UID: "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:44:55.055575 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055494    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" (UID: "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:44:55.055743 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055576    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-socket\") pod \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\" (UID: \"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6\") "
Jul 15 04:44:55.055743 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055698    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" (UID: "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:44:55.056107 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.055960    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" (UID: "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:44:55.056251 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.056222    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-data" (OuterVolumeSpecName: "istio-data") pod "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" (UID: "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:44:55.056366 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.056347    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:55.056431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.056371    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:55.056431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.056386    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:55.056431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.056398    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:55.056431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.056411    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:55.058484 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.058425    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-kube-api-access-t6wxg" (OuterVolumeSpecName: "kube-api-access-t6wxg") pod "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" (UID: "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6"). InnerVolumeSpecName "kube-api-access-t6wxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:44:55.058712 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.058632    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" (UID: "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:44:55.058825 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.058764    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-token" (OuterVolumeSpecName: "istio-token") pod "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" (UID: "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:44:55.058889 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.058851    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" (UID: "9452a6e6-11d2-49b8-9f1f-54ede62c6ce6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:44:55.157351 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.157311    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:55.157351 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.157339    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:55.157351 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.157348    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6wxg\" (UniqueName: \"kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-kube-api-access-t6wxg\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:55.157351 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.157359    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:44:55.287043 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.287001    2524 generic.go:356] "Generic (PLEG): container finished" podID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerID="faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d" exitCode=0
Jul 15 04:44:55.287249 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.287060    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"
Jul 15 04:44:55.287249 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.287076    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" event={"ID":"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6","Type":"ContainerDied","Data":"faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d"}
Jul 15 04:44:55.287249 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.287134    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt" event={"ID":"9452a6e6-11d2-49b8-9f1f-54ede62c6ce6","Type":"ContainerDied","Data":"c72f857700bc094bd91c554f87572b2fe31af33143b1e5454d9586a8cd43f24b"}
Jul 15 04:44:55.287249 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.287156    2524 scope.go:117] "RemoveContainer" containerID="faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d"
Jul 15 04:44:55.297077 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.297057    2524 scope.go:117] "RemoveContainer" containerID="69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287"
Jul 15 04:44:55.306484 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.306462    2524 scope.go:117] "RemoveContainer" containerID="faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d"
Jul 15 04:44:55.306836 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:44:55.306808    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d\": container with ID starting with faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d not found: ID does not exist" containerID="faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d"
Jul 15 04:44:55.306916 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.306844    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d"} err="failed to get container status \"faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d\": rpc error: code = NotFound desc = could not find container \"faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d\": container with ID starting with faa13cc73723ca00eb87ec30ae79eb2837f669760032346d3736a3a4825a282d not found: ID does not exist"
Jul 15 04:44:55.306916 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.306864    2524 scope.go:117] "RemoveContainer" containerID="69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287"
Jul 15 04:44:55.307125 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:44:55.307106    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287\": container with ID starting with 69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287 not found: ID does not exist" containerID="69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287"
Jul 15 04:44:55.307170 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.307133    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287"} err="failed to get container status \"69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287\": rpc error: code = NotFound desc = could not find container \"69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287\": container with ID starting with 69244707e849511b72388aa73ec40fa5603e3e62595f0882d5e2def6d1749287 not found: ID does not exist"
Jul 15 04:44:55.310741 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.310710    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"]
Jul 15 04:44:55.313823 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.313791    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-005cfa-openshift-default-5cc8db5b6c-dhrjt"]
Jul 15 04:44:55.433107 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:44:55.433068    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" path="/var/lib/kubelet/pods/9452a6e6-11d2-49b8-9f1f-54ede62c6ce6/volumes"
Jul 15 04:45:00.141565 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.141518    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734845-9k98d"]
Jul 15 04:45:00.142007 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.141963    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63db4cd3-59ce-4c36-9968-d62b7e7165b8" containerName="maas-api"
Jul 15 04:45:00.142007 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.141976    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="63db4cd3-59ce-4c36-9968-d62b7e7165b8" containerName="maas-api"
Jul 15 04:45:00.142007 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.141998    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerName="istio-proxy"
Jul 15 04:45:00.142007 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.142004    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerName="istio-proxy"
Jul 15 04:45:00.142141 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.142020    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerName="istio-proxy"
Jul 15 04:45:00.142141 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.142026    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerName="istio-proxy"
Jul 15 04:45:00.142141 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.142095    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="63db4cd3-59ce-4c36-9968-d62b7e7165b8" containerName="maas-api"
Jul 15 04:45:00.142141 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.142103    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerName="istio-proxy"
Jul 15 04:45:00.142141 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.142110    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="9452a6e6-11d2-49b8-9f1f-54ede62c6ce6" containerName="istio-proxy"
Jul 15 04:45:00.146971 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.146948    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734845-9k98d"
Jul 15 04:45:00.149579 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.149554    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jc2nz\""
Jul 15 04:45:00.155218 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.155185    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734845-9k98d"]
Jul 15 04:45:00.160333 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.160308    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734845-9k98d"
Jul 15 04:45:00.313387 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.313359    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734845-9k98d"]
Jul 15 04:45:00.315199 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:45:00.315161    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494a8ea8_2eee_42b2_b649_a9ebd283dd8a.slice/crio-d05b301ff24dc8017337f3d3d4559a8c31ab1c97106524132d529223882ebf70 WatchSource:0}: Error finding container d05b301ff24dc8017337f3d3d4559a8c31ab1c97106524132d529223882ebf70: Status 404 returned error can't find the container with id d05b301ff24dc8017337f3d3d4559a8c31ab1c97106524132d529223882ebf70
Jul 15 04:45:00.629467 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.629430    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"]
Jul 15 04:45:00.629772 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:00.629723    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7" podUID="3326dc69-29e0-4029-b619-b6c722d0b93b" containerName="maas-api" containerID="cri-o://5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1" gracePeriod=30
Jul 15 04:45:01.074860 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.074830    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:45:01.117679 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.117628    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3326dc69-29e0-4029-b619-b6c722d0b93b-openshift-service-ca\") pod \"3326dc69-29e0-4029-b619-b6c722d0b93b\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") "
Jul 15 04:45:01.117885 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.117743    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3326dc69-29e0-4029-b619-b6c722d0b93b-maas-api-tls\") pod \"3326dc69-29e0-4029-b619-b6c722d0b93b\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") "
Jul 15 04:45:01.117885 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.117806    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rld7z\" (UniqueName: \"kubernetes.io/projected/3326dc69-29e0-4029-b619-b6c722d0b93b-kube-api-access-rld7z\") pod \"3326dc69-29e0-4029-b619-b6c722d0b93b\" (UID: \"3326dc69-29e0-4029-b619-b6c722d0b93b\") "
Jul 15 04:45:01.118012 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.117987    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3326dc69-29e0-4029-b619-b6c722d0b93b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3326dc69-29e0-4029-b619-b6c722d0b93b" (UID: "3326dc69-29e0-4029-b619-b6c722d0b93b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:45:01.118137 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.118121    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3326dc69-29e0-4029-b619-b6c722d0b93b-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:01.119946 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.119920    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3326dc69-29e0-4029-b619-b6c722d0b93b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3326dc69-29e0-4029-b619-b6c722d0b93b" (UID: "3326dc69-29e0-4029-b619-b6c722d0b93b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:45:01.120051 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.120010    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3326dc69-29e0-4029-b619-b6c722d0b93b-kube-api-access-rld7z" (OuterVolumeSpecName: "kube-api-access-rld7z") pod "3326dc69-29e0-4029-b619-b6c722d0b93b" (UID: "3326dc69-29e0-4029-b619-b6c722d0b93b"). InnerVolumeSpecName "kube-api-access-rld7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:45:01.219129 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.219076    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3326dc69-29e0-4029-b619-b6c722d0b93b-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:01.219129 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.219121    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rld7z\" (UniqueName: \"kubernetes.io/projected/3326dc69-29e0-4029-b619-b6c722d0b93b-kube-api-access-rld7z\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:01.320518 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.320480    2524 generic.go:356] "Generic (PLEG): container finished" podID="494a8ea8-2eee-42b2-b649-a9ebd283dd8a" containerID="9cb39e012e2ab6ae2d254a35131668a5d276dfa276fcb4541f0c9c99f58336fc" exitCode=0
Jul 15 04:45:01.320741 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.320573    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734845-9k98d" event={"ID":"494a8ea8-2eee-42b2-b649-a9ebd283dd8a","Type":"ContainerDied","Data":"9cb39e012e2ab6ae2d254a35131668a5d276dfa276fcb4541f0c9c99f58336fc"}
Jul 15 04:45:01.320741 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.320656    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734845-9k98d" event={"ID":"494a8ea8-2eee-42b2-b649-a9ebd283dd8a","Type":"ContainerStarted","Data":"d05b301ff24dc8017337f3d3d4559a8c31ab1c97106524132d529223882ebf70"}
Jul 15 04:45:01.321947 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.321919    2524 generic.go:356] "Generic (PLEG): container finished" podID="3326dc69-29e0-4029-b619-b6c722d0b93b" containerID="5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1" exitCode=0
Jul 15 04:45:01.322094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.321966    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"
Jul 15 04:45:01.322094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.322006    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7" event={"ID":"3326dc69-29e0-4029-b619-b6c722d0b93b","Type":"ContainerDied","Data":"5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1"}
Jul 15 04:45:01.322094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.322058    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7" event={"ID":"3326dc69-29e0-4029-b619-b6c722d0b93b","Type":"ContainerDied","Data":"9b9e6c3c270de8af74e2f809f3746ed36b46a1f554a5827cc043d36e254bd539"}
Jul 15 04:45:01.322094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.322083    2524 scope.go:117] "RemoveContainer" containerID="5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1"
Jul 15 04:45:01.332104 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.332085    2524 scope.go:117] "RemoveContainer" containerID="5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1"
Jul 15 04:45:01.332448 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:45:01.332429    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1\": container with ID starting with 5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1 not found: ID does not exist" containerID="5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1"
Jul 15 04:45:01.332488 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.332458    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1"} err="failed to get container status \"5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1\": rpc error: code = NotFound desc = could not find container \"5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1\": container with ID starting with 5d71b4c1154ec34feef22ba9d1b0095d6a34576a16d1fe5cd0f7a650054b8db1 not found: ID does not exist"
Jul 15 04:45:01.350876 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.350837    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"]
Jul 15 04:45:01.356315 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.356279    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-194ff1-6df9c968d6-hxbp7"]
Jul 15 04:45:01.432190 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:01.432106    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3326dc69-29e0-4029-b619-b6c722d0b93b" path="/var/lib/kubelet/pods/3326dc69-29e0-4029-b619-b6c722d0b93b/volumes"
Jul 15 04:45:02.479345 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:02.479312    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734845-9k98d"
Jul 15 04:45:03.334369 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:03.334289    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734845-9k98d"
Jul 15 04:45:03.334369 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:03.334312    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734845-9k98d" event={"ID":"494a8ea8-2eee-42b2-b649-a9ebd283dd8a","Type":"ContainerDied","Data":"d05b301ff24dc8017337f3d3d4559a8c31ab1c97106524132d529223882ebf70"}
Jul 15 04:45:03.334575 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:03.334395    2524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d05b301ff24dc8017337f3d3d4559a8c31ab1c97106524132d529223882ebf70"
Jul 15 04:45:15.937827 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:15.937784    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"]
Jul 15 04:45:15.938350 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:15.938086    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy" containerID="cri-o://4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80" gracePeriod=30
Jul 15 04:45:16.885576 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.885543    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"]
Jul 15 04:45:16.885960 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.885946    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="494a8ea8-2eee-42b2-b649-a9ebd283dd8a" containerName="cleanup"
Jul 15 04:45:16.886007 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.885962    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="494a8ea8-2eee-42b2-b649-a9ebd283dd8a" containerName="cleanup"
Jul 15 04:45:16.886007 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.885985    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3326dc69-29e0-4029-b619-b6c722d0b93b" containerName="maas-api"
Jul 15 04:45:16.886007 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.885991    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="3326dc69-29e0-4029-b619-b6c722d0b93b" containerName="maas-api"
Jul 15 04:45:16.886119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.886041    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="3326dc69-29e0-4029-b619-b6c722d0b93b" containerName="maas-api"
Jul 15 04:45:16.886119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.886053    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="494a8ea8-2eee-42b2-b649-a9ebd283dd8a" containerName="cleanup"
Jul 15 04:45:16.888945 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.888923    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:16.891409 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.891386    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4aabeb-openshift-default-dockercfg-kbg5x\""
Jul 15 04:45:16.901234 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.901210    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"]
Jul 15 04:45:16.977298 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.977264    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-certs\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:16.977298 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.977308    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-envoy\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:16.977887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.977391    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc68z\" (UniqueName: \"kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-kube-api-access-tc68z\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:16.977887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.977425    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64aabc1f-bf19-4233-ac12-a3524f5537d3-istiod-ca-cert\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:16.977887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.977451    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-token\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:16.977887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.977567    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-credential-socket\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:16.977887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.977611    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-data\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:16.977887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.977650    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-socket\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:16.977887 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:16.977671    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-podinfo\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.078489 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.078451    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc68z\" (UniqueName: \"kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-kube-api-access-tc68z\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.078489 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.078493    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64aabc1f-bf19-4233-ac12-a3524f5537d3-istiod-ca-cert\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.078834 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.078514    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-token\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.078834 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.078579    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-credential-socket\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.078834 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.078751    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-data\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.078834 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.078806    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-socket\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.078834 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.078832    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-podinfo\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.079092 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.078890    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-certs\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.079092 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.078923    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-envoy\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.079092 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.079039    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-credential-socket\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.079255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.079132    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-data\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.079310 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.079271    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-socket\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.079310 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.079284    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-certs\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.079399 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.079377    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64aabc1f-bf19-4233-ac12-a3524f5537d3-istiod-ca-cert\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.081949 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.081917    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-envoy\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.082119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.081967    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-podinfo\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.093121 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.093092    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-token\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.093588 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.093567    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc68z\" (UniqueName: \"kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-kube-api-access-tc68z\") pod \"e2e-api-a-4aabeb-openshift-default-587f96565-txjrv\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") " pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.200997 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.200895    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:17.350219 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.350177    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"]
Jul 15 04:45:17.351402 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:45:17.351370    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64aabc1f_bf19_4233_ac12_a3524f5537d3.slice/crio-5b2e810b01de2176b75eb696b37af641343bbd696f4a35ef44e526a1608e03a5 WatchSource:0}: Error finding container 5b2e810b01de2176b75eb696b37af641343bbd696f4a35ef44e526a1608e03a5: Status 404 returned error can't find the container with id 5b2e810b01de2176b75eb696b37af641343bbd696f4a35ef44e526a1608e03a5
Jul 15 04:45:17.356572 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.356531    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:17.356709 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.356635    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:17.356709 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.356668    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:17.407830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:17.407797    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" event={"ID":"64aabc1f-bf19-4233-ac12-a3524f5537d3","Type":"ContainerStarted","Data":"5b2e810b01de2176b75eb696b37af641343bbd696f4a35ef44e526a1608e03a5"}
Jul 15 04:45:18.414448 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:18.414416    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" event={"ID":"64aabc1f-bf19-4233-ac12-a3524f5537d3","Type":"ContainerStarted","Data":"23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a"}
Jul 15 04:45:18.436862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:18.436806    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" podStartSLOduration=2.436790116 podStartE2EDuration="2.436790116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:45:18.433945254 +0000 UTC m=+2347.745326506" watchObservedRunningTime="2026-07-15 04:45:18.436790116 +0000 UTC m=+2347.748171363"
Jul 15 04:45:19.201329 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:19.201292    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:19.206559 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:19.206536    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:19.423901 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:19.423866    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:19.425139 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:19.425114    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:20.458375 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.458334    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"]
Jul 15 04:45:20.461916 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.461895    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:20.464712 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.464686    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-4aabeb-serving-cert\""
Jul 15 04:45:20.473991 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.473463    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"]
Jul 15 04:45:20.615924 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.615876    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/391123fc-2e84-4f13-9dc4-f9c0257084b9-openshift-service-ca\") pod \"maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:20.616124 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.615938    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t4mp\" (UniqueName: \"kubernetes.io/projected/391123fc-2e84-4f13-9dc4-f9c0257084b9-kube-api-access-8t4mp\") pod \"maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:20.616124 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.616047    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/391123fc-2e84-4f13-9dc4-f9c0257084b9-maas-api-tls\") pod \"maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:20.717290 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.717199    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/391123fc-2e84-4f13-9dc4-f9c0257084b9-openshift-service-ca\") pod \"maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:20.717290 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.717249    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4mp\" (UniqueName: \"kubernetes.io/projected/391123fc-2e84-4f13-9dc4-f9c0257084b9-kube-api-access-8t4mp\") pod \"maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:20.717545 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.717296    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/391123fc-2e84-4f13-9dc4-f9c0257084b9-maas-api-tls\") pod \"maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:20.717884 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.717859    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/391123fc-2e84-4f13-9dc4-f9c0257084b9-openshift-service-ca\") pod \"maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:20.719926 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.719903    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/391123fc-2e84-4f13-9dc4-f9c0257084b9-maas-api-tls\") pod \"maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:20.726138 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.726110    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t4mp\" (UniqueName: \"kubernetes.io/projected/391123fc-2e84-4f13-9dc4-f9c0257084b9-kube-api-access-8t4mp\") pod \"maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:20.778034 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.777991    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:20.934929 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:20.934892    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"]
Jul 15 04:45:20.936659 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:45:20.936626    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod391123fc_2e84_4f13_9dc4_f9c0257084b9.slice/crio-4d93a262c58ffee8bb9ddc0d417270a40116eb83691d89f8ecabfb20721abd47 WatchSource:0}: Error finding container 4d93a262c58ffee8bb9ddc0d417270a40116eb83691d89f8ecabfb20721abd47: Status 404 returned error can't find the container with id 4d93a262c58ffee8bb9ddc0d417270a40116eb83691d89f8ecabfb20721abd47
Jul 15 04:45:21.201079 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.201050    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:45:21.323524 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.323441    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x759d\" (UniqueName: \"kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-kube-api-access-x759d\") pod \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") "
Jul 15 04:45:21.323524 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.323506    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-envoy\") pod \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") "
Jul 15 04:45:21.323795 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.323549    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-socket\") pod \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") "
Jul 15 04:45:21.323795 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.323580    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-podinfo\") pod \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") "
Jul 15 04:45:21.323795 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.323629    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-credential-socket\") pod \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") "
Jul 15 04:45:21.323795 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.323660    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-data\") pod \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") "
Jul 15 04:45:21.323795 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.323690    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-token\") pod \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") "
Jul 15 04:45:21.323795 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.323746    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-certs\") pod \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") "
Jul 15 04:45:21.324098 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.323813    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istiod-ca-cert\") pod \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\" (UID: \"1ac3931d-6a31-4b84-862a-07d8e1f4020c\") "
Jul 15 04:45:21.324098 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.323904    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ac3931d-6a31-4b84-862a-07d8e1f4020c" (UID: "1ac3931d-6a31-4b84-862a-07d8e1f4020c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:45:21.324098 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.323963    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ac3931d-6a31-4b84-862a-07d8e1f4020c" (UID: "1ac3931d-6a31-4b84-862a-07d8e1f4020c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:45:21.324253 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.324198    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ac3931d-6a31-4b84-862a-07d8e1f4020c" (UID: "1ac3931d-6a31-4b84-862a-07d8e1f4020c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:45:21.324253 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.324238    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:21.324344 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.324260    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:21.324446 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.324424    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ac3931d-6a31-4b84-862a-07d8e1f4020c" (UID: "1ac3931d-6a31-4b84-862a-07d8e1f4020c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:45:21.325119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.325086    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ac3931d-6a31-4b84-862a-07d8e1f4020c" (UID: "1ac3931d-6a31-4b84-862a-07d8e1f4020c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:45:21.326005 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.325981    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ac3931d-6a31-4b84-862a-07d8e1f4020c" (UID: "1ac3931d-6a31-4b84-862a-07d8e1f4020c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:45:21.326005 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.325996    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-kube-api-access-x759d" (OuterVolumeSpecName: "kube-api-access-x759d") pod "1ac3931d-6a31-4b84-862a-07d8e1f4020c" (UID: "1ac3931d-6a31-4b84-862a-07d8e1f4020c"). InnerVolumeSpecName "kube-api-access-x759d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:45:21.326131 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.326003    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ac3931d-6a31-4b84-862a-07d8e1f4020c" (UID: "1ac3931d-6a31-4b84-862a-07d8e1f4020c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:45:21.326562 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.326546    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ac3931d-6a31-4b84-862a-07d8e1f4020c" (UID: "1ac3931d-6a31-4b84-862a-07d8e1f4020c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:45:21.425492 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.425457    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:21.425492 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.425488    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:21.425492 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.425497    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:21.425782 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.425505    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:21.425782 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.425512    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ac3931d-6a31-4b84-862a-07d8e1f4020c-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:21.425782 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.425519    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ac3931d-6a31-4b84-862a-07d8e1f4020c-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:21.425782 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.425527    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x759d\" (UniqueName: \"kubernetes.io/projected/1ac3931d-6a31-4b84-862a-07d8e1f4020c-kube-api-access-x759d\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:21.433809 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.433770    2524 generic.go:356] "Generic (PLEG): container finished" podID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerID="4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80" exitCode=0
Jul 15 04:45:21.433999 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.433819    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"
Jul 15 04:45:21.433999 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.433852    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" event={"ID":"1ac3931d-6a31-4b84-862a-07d8e1f4020c","Type":"ContainerDied","Data":"4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80"}
Jul 15 04:45:21.433999 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.433930    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w" event={"ID":"1ac3931d-6a31-4b84-862a-07d8e1f4020c","Type":"ContainerDied","Data":"eed0c9850a4613b80e4495d18986de95a86b765dc7ca604d69268fd9ebd152d9"}
Jul 15 04:45:21.433999 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.433962    2524 scope.go:117] "RemoveContainer" containerID="4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80"
Jul 15 04:45:21.435225 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.435201    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr" event={"ID":"391123fc-2e84-4f13-9dc4-f9c0257084b9","Type":"ContainerStarted","Data":"4d93a262c58ffee8bb9ddc0d417270a40116eb83691d89f8ecabfb20721abd47"}
Jul 15 04:45:21.450150 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.450128    2524 scope.go:117] "RemoveContainer" containerID="d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097"
Jul 15 04:45:21.459205 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.459170    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"]
Jul 15 04:45:21.464203 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.464159    2524 scope.go:117] "RemoveContainer" containerID="4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80"
Jul 15 04:45:21.464575 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:45:21.464549    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80\": container with ID starting with 4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80 not found: ID does not exist" containerID="4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80"
Jul 15 04:45:21.464710 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.464679    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80"} err="failed to get container status \"4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80\": rpc error: code = NotFound desc = could not find container \"4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80\": container with ID starting with 4fa1acd2cb16a9e2930176947fb4ec3eee0c489bbe89a3cd0db1cefe1ed9fc80 not found: ID does not exist"
Jul 15 04:45:21.464788 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.464720    2524 scope.go:117] "RemoveContainer" containerID="d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097"
Jul 15 04:45:21.465073 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:45:21.465049    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097\": container with ID starting with d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097 not found: ID does not exist" containerID="d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097"
Jul 15 04:45:21.465140 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.465088    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097"} err="failed to get container status \"d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097\": rpc error: code = NotFound desc = could not find container \"d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097\": container with ID starting with d6b7425bf621623ce51b4cb29e07a059280bc06c3463c63603149f39bd88a097 not found: ID does not exist"
Jul 15 04:45:21.465412 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:21.465394    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-194ff1-openshift-default-7b99c4c45c-c894w"]
Jul 15 04:45:22.442868 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:22.442816    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr" event={"ID":"391123fc-2e84-4f13-9dc4-f9c0257084b9","Type":"ContainerStarted","Data":"889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d"}
Jul 15 04:45:22.443073 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:22.442904    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:23.434318 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:23.434278    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" path="/var/lib/kubelet/pods/1ac3931d-6a31-4b84-862a-07d8e1f4020c/volumes"
Jul 15 04:45:25.691922 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.691854    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr" podStartSLOduration=5.16641671 podStartE2EDuration="5.691800291s" totalImagesPullingTime="525.383581ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:45:22.46082812 +0000 UTC m=+2351.772209368" watchObservedRunningTime="2026-07-15 04:45:25.691800291 +0000 UTC m=+2355.003181540"
Jul 15 04:45:25.692674 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.692647    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"]
Jul 15 04:45:25.693200 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.693184    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy"
Jul 15 04:45:25.693263 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.693201    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy"
Jul 15 04:45:25.693319 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.693308    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy"
Jul 15 04:45:25.693351 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.693323    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy"
Jul 15 04:45:25.697962 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.697941    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.700438 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.700412    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0135c8-openshift-default-dockercfg-vcswf\""
Jul 15 04:45:25.709893 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.709861    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"]
Jul 15 04:45:25.766351 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.766300    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-credential-socket\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.766585 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.766377    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-token\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.766585 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.766405    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-socket\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.766585 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.766484    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-podinfo\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.766585 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.766549    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-envoy\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.766830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.766582    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-data\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.766830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.766668    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n55vd\" (UniqueName: \"kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-kube-api-access-n55vd\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.766830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.766707    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c00532-bfa2-491b-80ff-25555a83cb2a-istiod-ca-cert\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.766830 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.766735    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-certs\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.867992 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.867945    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n55vd\" (UniqueName: \"kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-kube-api-access-n55vd\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.868226 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.867997    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c00532-bfa2-491b-80ff-25555a83cb2a-istiod-ca-cert\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.868226 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.868047    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-certs\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.868226 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.868083    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-credential-socket\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.868226 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.868116    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-token\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.868226 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.868143    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-socket\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.868226 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.868216    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-podinfo\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.868545 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.868337    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-envoy\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.868545 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.868379    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-data\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.868696 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.868536    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-certs\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.868696 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.868565    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-credential-socket\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.868868 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.868843    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-socket\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.869246 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.869194    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-data\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.869396 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.869314    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c00532-bfa2-491b-80ff-25555a83cb2a-istiod-ca-cert\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.871455 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.871421    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-envoy\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.871664 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.871645    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-podinfo\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.876053 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.876031    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-token\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:25.876224 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:25.876201    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n55vd\" (UniqueName: \"kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-kube-api-access-n55vd\") pod \"e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") " pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:26.011460 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:26.011354    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:26.206034 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:26.205998    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"]
Jul 15 04:45:26.206958 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:45:26.206917    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c00532_bfa2_491b_80ff_25555a83cb2a.slice/crio-96face83662fd6172fc34fb04fbf8e596b1b1468192439b4b531ea3737452ce6 WatchSource:0}: Error finding container 96face83662fd6172fc34fb04fbf8e596b1b1468192439b4b531ea3737452ce6: Status 404 returned error can't find the container with id 96face83662fd6172fc34fb04fbf8e596b1b1468192439b4b531ea3737452ce6
Jul 15 04:45:26.209222 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:26.209188    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:26.209301 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:26.209258    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:26.209301 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:26.209294    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:26.464631 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:26.464572    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" event={"ID":"61c00532-bfa2-491b-80ff-25555a83cb2a","Type":"ContainerStarted","Data":"c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c"}
Jul 15 04:45:26.464842 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:26.464659    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" event={"ID":"61c00532-bfa2-491b-80ff-25555a83cb2a","Type":"ContainerStarted","Data":"96face83662fd6172fc34fb04fbf8e596b1b1468192439b4b531ea3737452ce6"}
Jul 15 04:45:26.483783 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:26.483718    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" podStartSLOduration=1.483696841 podStartE2EDuration="1.483696841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:45:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:45:26.483426589 +0000 UTC m=+2355.794807837" watchObservedRunningTime="2026-07-15 04:45:26.483696841 +0000 UTC m=+2355.795078088"
Jul 15 04:45:27.011817 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:27.011767    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:27.017505 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:27.017476    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:27.472076 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:27.472044    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:27.478763 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:27.478730    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:28.454026 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.453987    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:28.641942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.641901    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"]
Jul 15 04:45:28.642295 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.642282    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy"
Jul 15 04:45:28.642354 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.642296    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ac3931d-6a31-4b84-862a-07d8e1f4020c" containerName="istio-proxy"
Jul 15 04:45:28.645434 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.645415    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:28.647787 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.647764    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0135c8-serving-cert\""
Jul 15 04:45:28.654063 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.654040    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"]
Jul 15 04:45:28.700217 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.700170    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mdzf\" (UniqueName: \"kubernetes.io/projected/2119af63-82b4-4bad-a480-837d8413558e-kube-api-access-6mdzf\") pod \"maas-api-e2e-api-b-0135c8-5989bf95b6-9942j\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:28.700426 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.700234    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2119af63-82b4-4bad-a480-837d8413558e-openshift-service-ca\") pod \"maas-api-e2e-api-b-0135c8-5989bf95b6-9942j\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:28.700426 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.700362    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2119af63-82b4-4bad-a480-837d8413558e-maas-api-tls\") pod \"maas-api-e2e-api-b-0135c8-5989bf95b6-9942j\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:28.802101 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.802002    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2119af63-82b4-4bad-a480-837d8413558e-maas-api-tls\") pod \"maas-api-e2e-api-b-0135c8-5989bf95b6-9942j\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:28.802278 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.802111    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mdzf\" (UniqueName: \"kubernetes.io/projected/2119af63-82b4-4bad-a480-837d8413558e-kube-api-access-6mdzf\") pod \"maas-api-e2e-api-b-0135c8-5989bf95b6-9942j\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:28.802278 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.802149    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2119af63-82b4-4bad-a480-837d8413558e-openshift-service-ca\") pod \"maas-api-e2e-api-b-0135c8-5989bf95b6-9942j\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:28.803272 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:45:28.802869    2524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-serving-cert: secret "maas-api-e2e-api-b-0135c8-serving-cert" not found
Jul 15 04:45:28.803272 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:45:28.803061    2524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2119af63-82b4-4bad-a480-837d8413558e-maas-api-tls podName:2119af63-82b4-4bad-a480-837d8413558e nodeName:}" failed. No retries permitted until 2026-07-15 04:45:29.303005582 +0000 UTC m=+2358.614386810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2119af63-82b4-4bad-a480-837d8413558e-maas-api-tls") pod "maas-api-e2e-api-b-0135c8-5989bf95b6-9942j" (UID: "2119af63-82b4-4bad-a480-837d8413558e") : secret "maas-api-e2e-api-b-0135c8-serving-cert" not found
Jul 15 04:45:28.803272 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.803066    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2119af63-82b4-4bad-a480-837d8413558e-openshift-service-ca\") pod \"maas-api-e2e-api-b-0135c8-5989bf95b6-9942j\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:28.812349 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:28.812322    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mdzf\" (UniqueName: \"kubernetes.io/projected/2119af63-82b4-4bad-a480-837d8413558e-kube-api-access-6mdzf\") pod \"maas-api-e2e-api-b-0135c8-5989bf95b6-9942j\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:29.306103 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:29.306056    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2119af63-82b4-4bad-a480-837d8413558e-maas-api-tls\") pod \"maas-api-e2e-api-b-0135c8-5989bf95b6-9942j\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:29.308917 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:29.308884    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2119af63-82b4-4bad-a480-837d8413558e-maas-api-tls\") pod \"maas-api-e2e-api-b-0135c8-5989bf95b6-9942j\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:29.557446 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:29.557349    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:29.749259 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:29.749223    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"]
Jul 15 04:45:29.751292 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:45:29.751247    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2119af63_82b4_4bad_a480_837d8413558e.slice/crio-8e8f3da3d29ad5378716b1346d961ec7b64ca3376a87da5c897eba340088963d WatchSource:0}: Error finding container 8e8f3da3d29ad5378716b1346d961ec7b64ca3376a87da5c897eba340088963d: Status 404 returned error can't find the container with id 8e8f3da3d29ad5378716b1346d961ec7b64ca3376a87da5c897eba340088963d
Jul 15 04:45:30.495871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:30.494911    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j" event={"ID":"2119af63-82b4-4bad-a480-837d8413558e","Type":"ContainerStarted","Data":"7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3"}
Jul 15 04:45:30.495871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:30.495008    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j" event={"ID":"2119af63-82b4-4bad-a480-837d8413558e","Type":"ContainerStarted","Data":"8e8f3da3d29ad5378716b1346d961ec7b64ca3376a87da5c897eba340088963d"}
Jul 15 04:45:30.495871 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:30.495812    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:30.513046 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:30.512989    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j" podStartSLOduration=2.187651672 podStartE2EDuration="2.512972927s" totalImagesPullingTime="325.321255ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:45:30.510553877 +0000 UTC m=+2359.821935126" watchObservedRunningTime="2026-07-15 04:45:30.512972927 +0000 UTC m=+2359.824354174"
Jul 15 04:45:34.425541 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:34.425503    2524 patch_prober.go:28] interesting pod/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 15 04:45:34.426001 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:34.425577    2524 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 15 04:45:34.517906 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:34.517867    2524 generic.go:356] "Generic (PLEG): container finished" podID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerID="23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a" exitCode=0
Jul 15 04:45:34.518088 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:34.517941    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" event={"ID":"64aabc1f-bf19-4233-ac12-a3524f5537d3","Type":"ContainerDied","Data":"23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a"}
Jul 15 04:45:34.518378 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:34.518362    2524 scope.go:117] "RemoveContainer" containerID="23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a"
Jul 15 04:45:34.519532 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:34.519501    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:34.519623 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:34.519584    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:34.519668 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:34.519654    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:35.524804 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:35.524771    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" event={"ID":"64aabc1f-bf19-4233-ac12-a3524f5537d3","Type":"ContainerStarted","Data":"a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa"}
Jul 15 04:45:36.202106 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:36.202064    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:36.202296 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:36.202270    2524 patch_prober.go:28] interesting pod/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 15 04:45:36.202342 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:36.202313    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 15 04:45:37.202744 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:37.202707    2524 patch_prober.go:28] interesting pod/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 15 04:45:37.203394 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:37.203348    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 15 04:45:37.512457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:37.512364    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:45:38.202383 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:38.202342    2524 patch_prober.go:28] interesting pod/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 15 04:45:38.202738 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:38.202702    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 15 04:45:39.201892 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:39.201849    2524 patch_prober.go:28] interesting pod/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 15 04:45:39.202522 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:39.202481    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 15 04:45:40.205979 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:40.205938    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:40.206396 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:40.206360    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:40.207259 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:40.207237    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:45:43.513829 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:43.513783    2524 patch_prober.go:28] interesting pod/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 15 04:45:43.514349 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:43.513869    2524 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 15 04:45:44.575353 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:44.575243    2524 generic.go:356] "Generic (PLEG): container finished" podID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerID="c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c" exitCode=0
Jul 15 04:45:44.575353 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:44.575313    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" event={"ID":"61c00532-bfa2-491b-80ff-25555a83cb2a","Type":"ContainerDied","Data":"c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c"}
Jul 15 04:45:44.575858 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:44.575839    2524 scope.go:117] "RemoveContainer" containerID="c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c"
Jul 15 04:45:44.577022 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:44.576993    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:44.577088 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:44.577066    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:44.577126 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:44.577102    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:45:45.587238 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:45.587199    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" event={"ID":"61c00532-bfa2-491b-80ff-25555a83cb2a","Type":"ContainerStarted","Data":"ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f"}
Jul 15 04:45:46.011503 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:46.011458    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:46.011872 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:46.011849    2524 patch_prober.go:28] interesting pod/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 15 04:45:46.011928 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:46.011899    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 15 04:45:47.012480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:47.012436    2524 patch_prober.go:28] interesting pod/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 15 04:45:47.012952 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:47.012498    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 15 04:45:48.012411 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:48.012365    2524 patch_prober.go:28] interesting pod/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 15 04:45:48.012640 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:48.012426    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 15 04:45:49.012016 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:49.011969    2524 patch_prober.go:28] interesting pod/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 15 04:45:49.012249 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:49.012038    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 15 04:45:50.015590 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:50.015558    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:50.016099 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:50.016065    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:50.016958 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:50.016937    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:45:51.653856 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:51.653812    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"]
Jul 15 04:45:51.654380 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:51.654131    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr" podUID="391123fc-2e84-4f13-9dc4-f9c0257084b9" containerName="maas-api" containerID="cri-o://889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d" gracePeriod=30
Jul 15 04:45:51.915804 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:51.915737    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:52.041834 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.041790    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/391123fc-2e84-4f13-9dc4-f9c0257084b9-openshift-service-ca\") pod \"391123fc-2e84-4f13-9dc4-f9c0257084b9\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") "
Jul 15 04:45:52.042030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.041897    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/391123fc-2e84-4f13-9dc4-f9c0257084b9-maas-api-tls\") pod \"391123fc-2e84-4f13-9dc4-f9c0257084b9\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") "
Jul 15 04:45:52.042030 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.041965    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t4mp\" (UniqueName: \"kubernetes.io/projected/391123fc-2e84-4f13-9dc4-f9c0257084b9-kube-api-access-8t4mp\") pod \"391123fc-2e84-4f13-9dc4-f9c0257084b9\" (UID: \"391123fc-2e84-4f13-9dc4-f9c0257084b9\") "
Jul 15 04:45:52.042261 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.042239    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/391123fc-2e84-4f13-9dc4-f9c0257084b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "391123fc-2e84-4f13-9dc4-f9c0257084b9" (UID: "391123fc-2e84-4f13-9dc4-f9c0257084b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:45:52.042320 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.042307    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/391123fc-2e84-4f13-9dc4-f9c0257084b9-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:52.044215 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.044185    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/391123fc-2e84-4f13-9dc4-f9c0257084b9-kube-api-access-8t4mp" (OuterVolumeSpecName: "kube-api-access-8t4mp") pod "391123fc-2e84-4f13-9dc4-f9c0257084b9" (UID: "391123fc-2e84-4f13-9dc4-f9c0257084b9"). InnerVolumeSpecName "kube-api-access-8t4mp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:45:52.044328 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.044195    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/391123fc-2e84-4f13-9dc4-f9c0257084b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "391123fc-2e84-4f13-9dc4-f9c0257084b9" (UID: "391123fc-2e84-4f13-9dc4-f9c0257084b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:45:52.143703 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.143655    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/391123fc-2e84-4f13-9dc4-f9c0257084b9-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:52.143703 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.143691    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t4mp\" (UniqueName: \"kubernetes.io/projected/391123fc-2e84-4f13-9dc4-f9c0257084b9-kube-api-access-8t4mp\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:45:52.624917 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.624874    2524 generic.go:356] "Generic (PLEG): container finished" podID="391123fc-2e84-4f13-9dc4-f9c0257084b9" containerID="889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d" exitCode=0
Jul 15 04:45:52.625113 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.624927    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"
Jul 15 04:45:52.625113 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.624964    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr" event={"ID":"391123fc-2e84-4f13-9dc4-f9c0257084b9","Type":"ContainerDied","Data":"889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d"}
Jul 15 04:45:52.625113 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.625024    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr" event={"ID":"391123fc-2e84-4f13-9dc4-f9c0257084b9","Type":"ContainerDied","Data":"4d93a262c58ffee8bb9ddc0d417270a40116eb83691d89f8ecabfb20721abd47"}
Jul 15 04:45:52.625113 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.625049    2524 scope.go:117] "RemoveContainer" containerID="889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d"
Jul 15 04:45:52.634422 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.634403    2524 scope.go:117] "RemoveContainer" containerID="889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d"
Jul 15 04:45:52.634767 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:45:52.634743    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d\": container with ID starting with 889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d not found: ID does not exist" containerID="889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d"
Jul 15 04:45:52.634864 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.634777    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d"} err="failed to get container status \"889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d\": rpc error: code = NotFound desc = could not find container \"889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d\": container with ID starting with 889b8457e7e8969100e1e21c7a657560949f792ac89533c98bbf068c1bfb6c9d not found: ID does not exist"
Jul 15 04:45:52.646480 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.646444    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"]
Jul 15 04:45:52.649923 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:52.649891    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4aabeb-59b74dfc74-sc9pr"]
Jul 15 04:45:53.432904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:45:53.432872    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="391123fc-2e84-4f13-9dc4-f9c0257084b9" path="/var/lib/kubelet/pods/391123fc-2e84-4f13-9dc4-f9c0257084b9/volumes"
Jul 15 04:46:06.920906 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:06.920869    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"]
Jul 15 04:46:06.921339 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:06.921096    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy" containerID="cri-o://a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa" gracePeriod=30
Jul 15 04:46:10.209465 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:10.209422    2524 patch_prober.go:28] interesting pod/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 04:46:10.209942 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:10.209490    2524 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 04:46:11.481310 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:11.481277    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:46:11.484307 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:11.484274    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:46:12.178891 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.178866    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:46:12.237833 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.237732    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-envoy\") pod \"64aabc1f-bf19-4233-ac12-a3524f5537d3\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") "
Jul 15 04:46:12.237833 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.237774    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-podinfo\") pod \"64aabc1f-bf19-4233-ac12-a3524f5537d3\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") "
Jul 15 04:46:12.237833 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.237806    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-token\") pod \"64aabc1f-bf19-4233-ac12-a3524f5537d3\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") "
Jul 15 04:46:12.238132 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.237847    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-certs\") pod \"64aabc1f-bf19-4233-ac12-a3524f5537d3\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") "
Jul 15 04:46:12.238132 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.237889    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-credential-socket\") pod \"64aabc1f-bf19-4233-ac12-a3524f5537d3\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") "
Jul 15 04:46:12.238132 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.237921    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc68z\" (UniqueName: \"kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-kube-api-access-tc68z\") pod \"64aabc1f-bf19-4233-ac12-a3524f5537d3\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") "
Jul 15 04:46:12.238132 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.237972    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-data\") pod \"64aabc1f-bf19-4233-ac12-a3524f5537d3\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") "
Jul 15 04:46:12.238132 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.237995    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-socket\") pod \"64aabc1f-bf19-4233-ac12-a3524f5537d3\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") "
Jul 15 04:46:12.238132 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.238043    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64aabc1f-bf19-4233-ac12-a3524f5537d3-istiod-ca-cert\") pod \"64aabc1f-bf19-4233-ac12-a3524f5537d3\" (UID: \"64aabc1f-bf19-4233-ac12-a3524f5537d3\") "
Jul 15 04:46:12.238437 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.238163    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "64aabc1f-bf19-4233-ac12-a3524f5537d3" (UID: "64aabc1f-bf19-4233-ac12-a3524f5537d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:46:12.238437 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.238294    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "64aabc1f-bf19-4233-ac12-a3524f5537d3" (UID: "64aabc1f-bf19-4233-ac12-a3524f5537d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:46:12.238552 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.238478    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:12.238552 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.238500    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:12.238686 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.238568    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64aabc1f-bf19-4233-ac12-a3524f5537d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "64aabc1f-bf19-4233-ac12-a3524f5537d3" (UID: "64aabc1f-bf19-4233-ac12-a3524f5537d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:46:12.239062 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.239034    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "64aabc1f-bf19-4233-ac12-a3524f5537d3" (UID: "64aabc1f-bf19-4233-ac12-a3524f5537d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:46:12.239691 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.239662    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "64aabc1f-bf19-4233-ac12-a3524f5537d3" (UID: "64aabc1f-bf19-4233-ac12-a3524f5537d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:46:12.240432 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.240402    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "64aabc1f-bf19-4233-ac12-a3524f5537d3" (UID: "64aabc1f-bf19-4233-ac12-a3524f5537d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:46:12.240536 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.240476    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "64aabc1f-bf19-4233-ac12-a3524f5537d3" (UID: "64aabc1f-bf19-4233-ac12-a3524f5537d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:46:12.240761 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.240740    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "64aabc1f-bf19-4233-ac12-a3524f5537d3" (UID: "64aabc1f-bf19-4233-ac12-a3524f5537d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:46:12.240831 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.240769    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-kube-api-access-tc68z" (OuterVolumeSpecName: "kube-api-access-tc68z") pod "64aabc1f-bf19-4233-ac12-a3524f5537d3" (UID: "64aabc1f-bf19-4233-ac12-a3524f5537d3"). InnerVolumeSpecName "kube-api-access-tc68z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:46:12.339870 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.339824    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:12.339870 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.339857    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:12.339870 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.339868    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64aabc1f-bf19-4233-ac12-a3524f5537d3-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:12.339870 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.339876    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:12.340152 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.339887    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:12.340152 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.339897    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:12.340152 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.339905    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc68z\" (UniqueName: \"kubernetes.io/projected/64aabc1f-bf19-4233-ac12-a3524f5537d3-kube-api-access-tc68z\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:12.727343 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.727306    2524 generic.go:356] "Generic (PLEG): container finished" podID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerID="a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa" exitCode=0
Jul 15 04:46:12.727343 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.727354    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"
Jul 15 04:46:12.727851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.727361    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" event={"ID":"64aabc1f-bf19-4233-ac12-a3524f5537d3","Type":"ContainerDied","Data":"a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa"}
Jul 15 04:46:12.727851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.727412    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv" event={"ID":"64aabc1f-bf19-4233-ac12-a3524f5537d3","Type":"ContainerDied","Data":"5b2e810b01de2176b75eb696b37af641343bbd696f4a35ef44e526a1608e03a5"}
Jul 15 04:46:12.727851 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.727434    2524 scope.go:117] "RemoveContainer" containerID="a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa"
Jul 15 04:46:12.738025 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.738006    2524 scope.go:117] "RemoveContainer" containerID="23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a"
Jul 15 04:46:12.747457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.747429    2524 scope.go:117] "RemoveContainer" containerID="a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa"
Jul 15 04:46:12.748153 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:46:12.748122    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa\": container with ID starting with a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa not found: ID does not exist" containerID="a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa"
Jul 15 04:46:12.748272 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.748167    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa"} err="failed to get container status \"a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa\": rpc error: code = NotFound desc = could not find container \"a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa\": container with ID starting with a37de3a56b4e8f3aac7383ceb62e310667cca70050572a0f594ba66006fddbaa not found: ID does not exist"
Jul 15 04:46:12.748272 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.748192    2524 scope.go:117] "RemoveContainer" containerID="23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a"
Jul 15 04:46:12.748561 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:46:12.748541    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a\": container with ID starting with 23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a not found: ID does not exist" containerID="23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a"
Jul 15 04:46:12.748635 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.748568    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a"} err="failed to get container status \"23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a\": rpc error: code = NotFound desc = could not find container \"23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a\": container with ID starting with 23f52d9be1deb3b80f3651b030c21c8fa5d33baeb4e848d0118d496183cba17a not found: ID does not exist"
Jul 15 04:46:12.750863 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.750748    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"]
Jul 15 04:46:12.756689 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:12.755543    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4aabeb-openshift-default-587f96565-txjrv"]
Jul 15 04:46:13.432429 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:13.432390    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" path="/var/lib/kubelet/pods/64aabc1f-bf19-4233-ac12-a3524f5537d3/volumes"
Jul 15 04:46:17.813519 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:17.813477    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"]
Jul 15 04:46:17.814106 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:17.813779    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j" podUID="2119af63-82b4-4bad-a480-837d8413558e" containerName="maas-api" containerID="cri-o://7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3" gracePeriod=30
Jul 15 04:46:18.070319 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.070251    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:46:18.093680 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.093638    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mdzf\" (UniqueName: \"kubernetes.io/projected/2119af63-82b4-4bad-a480-837d8413558e-kube-api-access-6mdzf\") pod \"2119af63-82b4-4bad-a480-837d8413558e\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") "
Jul 15 04:46:18.093860 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.093765    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2119af63-82b4-4bad-a480-837d8413558e-maas-api-tls\") pod \"2119af63-82b4-4bad-a480-837d8413558e\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") "
Jul 15 04:46:18.093928 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.093879    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2119af63-82b4-4bad-a480-837d8413558e-openshift-service-ca\") pod \"2119af63-82b4-4bad-a480-837d8413558e\" (UID: \"2119af63-82b4-4bad-a480-837d8413558e\") "
Jul 15 04:46:18.094298 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.094259    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2119af63-82b4-4bad-a480-837d8413558e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2119af63-82b4-4bad-a480-837d8413558e" (UID: "2119af63-82b4-4bad-a480-837d8413558e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:46:18.096006 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.095974    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2119af63-82b4-4bad-a480-837d8413558e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2119af63-82b4-4bad-a480-837d8413558e" (UID: "2119af63-82b4-4bad-a480-837d8413558e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:46:18.096127 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.096112    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2119af63-82b4-4bad-a480-837d8413558e-kube-api-access-6mdzf" (OuterVolumeSpecName: "kube-api-access-6mdzf") pod "2119af63-82b4-4bad-a480-837d8413558e" (UID: "2119af63-82b4-4bad-a480-837d8413558e"). InnerVolumeSpecName "kube-api-access-6mdzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:46:18.194660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.194588    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2119af63-82b4-4bad-a480-837d8413558e-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:18.194660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.194655    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2119af63-82b4-4bad-a480-837d8413558e-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:18.194660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.194666    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mdzf\" (UniqueName: \"kubernetes.io/projected/2119af63-82b4-4bad-a480-837d8413558e-kube-api-access-6mdzf\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:18.758555 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.758516    2524 generic.go:356] "Generic (PLEG): container finished" podID="2119af63-82b4-4bad-a480-837d8413558e" containerID="7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3" exitCode=0
Jul 15 04:46:18.758555 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.758567    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"
Jul 15 04:46:18.758895 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.758650    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j" event={"ID":"2119af63-82b4-4bad-a480-837d8413558e","Type":"ContainerDied","Data":"7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3"}
Jul 15 04:46:18.758895 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.758719    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j" event={"ID":"2119af63-82b4-4bad-a480-837d8413558e","Type":"ContainerDied","Data":"8e8f3da3d29ad5378716b1346d961ec7b64ca3376a87da5c897eba340088963d"}
Jul 15 04:46:18.758895 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.758742    2524 scope.go:117] "RemoveContainer" containerID="7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3"
Jul 15 04:46:18.768773 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.768750    2524 scope.go:117] "RemoveContainer" containerID="7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3"
Jul 15 04:46:18.769098 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:46:18.769079    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3\": container with ID starting with 7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3 not found: ID does not exist" containerID="7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3"
Jul 15 04:46:18.769157 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.769107    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3"} err="failed to get container status \"7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3\": rpc error: code = NotFound desc = could not find container \"7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3\": container with ID starting with 7ccb65d0f50cfdac7ff9abc1748ad73d94b4104d28e15c0317f13bbeedb830a3 not found: ID does not exist"
Jul 15 04:46:18.780575 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.780538    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"]
Jul 15 04:46:18.784271 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:18.784239    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0135c8-5989bf95b6-9942j"]
Jul 15 04:46:19.433079 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:19.433039    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2119af63-82b4-4bad-a480-837d8413558e" path="/var/lib/kubelet/pods/2119af63-82b4-4bad-a480-837d8413558e/volumes"
Jul 15 04:46:33.139022 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:33.138784    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"]
Jul 15 04:46:33.139479 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:33.139206    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy" containerID="cri-o://ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f" gracePeriod=30
Jul 15 04:46:34.069456 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.069414    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"]
Jul 15 04:46:34.070014 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.069996    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy"
Jul 15 04:46:34.070074 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.070017    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy"
Jul 15 04:46:34.070074 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.070030    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="391123fc-2e84-4f13-9dc4-f9c0257084b9" containerName="maas-api"
Jul 15 04:46:34.070074 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.070039    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="391123fc-2e84-4f13-9dc4-f9c0257084b9" containerName="maas-api"
Jul 15 04:46:34.070191 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.070078    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2119af63-82b4-4bad-a480-837d8413558e" containerName="maas-api"
Jul 15 04:46:34.070191 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.070088    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="2119af63-82b4-4bad-a480-837d8413558e" containerName="maas-api"
Jul 15 04:46:34.070191 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.070168    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy"
Jul 15 04:46:34.070191 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.070179    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="2119af63-82b4-4bad-a480-837d8413558e" containerName="maas-api"
Jul 15 04:46:34.070332 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.070191    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="391123fc-2e84-4f13-9dc4-f9c0257084b9" containerName="maas-api"
Jul 15 04:46:34.073680 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.073639    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.076109 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.076086    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-c2a8a9-openshift-default-dockercfg-r5xt4\""
Jul 15 04:46:34.085024 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.084996    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"]
Jul 15 04:46:34.151963 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.151916    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-socket\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.152440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.151979    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-data\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.152440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.152102    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9djd6\" (UniqueName: \"kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-kube-api-access-9djd6\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.152440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.152161    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-token\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.152440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.152192    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/528f9441-66ec-4cc0-9245-0fb56a98d014-istiod-ca-cert\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.152440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.152240    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-credential-socket\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.152440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.152278    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-podinfo\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.152440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.152308    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-certs\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.152440 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.152336    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-envoy\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.253298 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253259    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-credential-socket\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.253298 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253301    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-podinfo\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.253569 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253323    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-certs\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.253569 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253339    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-envoy\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.253569 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253374    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-socket\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.253569 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253404    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-data\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.253569 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253453    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9djd6\" (UniqueName: \"kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-kube-api-access-9djd6\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.253891 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253621    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-token\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.253891 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253664    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/528f9441-66ec-4cc0-9245-0fb56a98d014-istiod-ca-cert\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.253891 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253756    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-certs\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.253891 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253834    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-socket\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.254095 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.253941    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-data\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.254279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.254256    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/528f9441-66ec-4cc0-9245-0fb56a98d014-istiod-ca-cert\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.254497 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.254472    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-credential-socket\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.255904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.255885    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-envoy\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.256238 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.256221    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-podinfo\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.261521 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.261495    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-token\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.261743 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.261723    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9djd6\" (UniqueName: \"kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-kube-api-access-9djd6\") pod \"e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") " pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.386300 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.386256    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:34.534441 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.534361    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"]
Jul 15 04:46:34.537441 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:46:34.537396    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod528f9441_66ec_4cc0_9245_0fb56a98d014.slice/crio-c59ca656498bb0d95f279faf6fac005a869f65cbb3a1c98ec06a826fd4845ba6 WatchSource:0}: Error finding container c59ca656498bb0d95f279faf6fac005a869f65cbb3a1c98ec06a826fd4845ba6: Status 404 returned error can't find the container with id c59ca656498bb0d95f279faf6fac005a869f65cbb3a1c98ec06a826fd4845ba6
Jul 15 04:46:34.539888 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.539853    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:34.539966 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.539943    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:34.540012 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.539978    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:34.840325 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.840233    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" event={"ID":"528f9441-66ec-4cc0-9245-0fb56a98d014","Type":"ContainerStarted","Data":"c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe"}
Jul 15 04:46:34.840325 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.840313    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" event={"ID":"528f9441-66ec-4cc0-9245-0fb56a98d014","Type":"ContainerStarted","Data":"c59ca656498bb0d95f279faf6fac005a869f65cbb3a1c98ec06a826fd4845ba6"}
Jul 15 04:46:34.860971 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:34.860914    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" podStartSLOduration=0.860894561 podStartE2EDuration="860.894561ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:46:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:46:34.85888851 +0000 UTC m=+2424.170269772" watchObservedRunningTime="2026-07-15 04:46:34.860894561 +0000 UTC m=+2424.172275807"
Jul 15 04:46:35.018540 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:35.018498    2524 patch_prober.go:28] interesting pod/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 04:46:35.018725 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:35.018555    2524 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 04:46:35.387221 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:35.387183    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:35.392904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:35.392871    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:35.845404 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:35.845319    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:35.846295 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:35.846275    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:37.025286 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.025249    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"]
Jul 15 04:46:37.025854 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.025837    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy"
Jul 15 04:46:37.025913 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.025858    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy"
Jul 15 04:46:37.025994 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.025982    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="64aabc1f-bf19-4233-ac12-a3524f5537d3" containerName="istio-proxy"
Jul 15 04:46:37.029409 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.029387    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:37.032047 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.032017    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-c2a8a9-serving-cert\""
Jul 15 04:46:37.038869 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.038843    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"]
Jul 15 04:46:37.183660 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.183579    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc388544-d033-4f50-a415-620b7e658220-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:37.183867 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.183692    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc388544-d033-4f50-a415-620b7e658220-maas-api-tls\") pod \"maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:37.183867 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.183740    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb6bj\" (UniqueName: \"kubernetes.io/projected/fc388544-d033-4f50-a415-620b7e658220-kube-api-access-hb6bj\") pod \"maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:37.284992 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.284899    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc388544-d033-4f50-a415-620b7e658220-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:37.284992 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.284946    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc388544-d033-4f50-a415-620b7e658220-maas-api-tls\") pod \"maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:37.285202 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.285010    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb6bj\" (UniqueName: \"kubernetes.io/projected/fc388544-d033-4f50-a415-620b7e658220-kube-api-access-hb6bj\") pod \"maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:37.285681 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.285653    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc388544-d033-4f50-a415-620b7e658220-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:37.287722 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.287694    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc388544-d033-4f50-a415-620b7e658220-maas-api-tls\") pod \"maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:37.295465 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.295426    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb6bj\" (UniqueName: \"kubernetes.io/projected/fc388544-d033-4f50-a415-620b7e658220-kube-api-access-hb6bj\") pod \"maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:37.345163 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.345113    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:37.495554 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.495525    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"]
Jul 15 04:46:37.497803 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:46:37.497767    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc388544_d033_4f50_a415_620b7e658220.slice/crio-61cd3eb8882950f511c2d8384ffe735b6b5be482e65c4a725f22a756156b60f2 WatchSource:0}: Error finding container 61cd3eb8882950f511c2d8384ffe735b6b5be482e65c4a725f22a756156b60f2: Status 404 returned error can't find the container with id 61cd3eb8882950f511c2d8384ffe735b6b5be482e65c4a725f22a756156b60f2
Jul 15 04:46:37.856680 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:37.856641    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f" event={"ID":"fc388544-d033-4f50-a415-620b7e658220","Type":"ContainerStarted","Data":"61cd3eb8882950f511c2d8384ffe735b6b5be482e65c4a725f22a756156b60f2"}
Jul 15 04:46:38.406808 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.406737    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:46:38.497088 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497042    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-socket\") pod \"61c00532-bfa2-491b-80ff-25555a83cb2a\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") "
Jul 15 04:46:38.497088 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497093    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-data\") pod \"61c00532-bfa2-491b-80ff-25555a83cb2a\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") "
Jul 15 04:46:38.497399 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497131    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-podinfo\") pod \"61c00532-bfa2-491b-80ff-25555a83cb2a\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") "
Jul 15 04:46:38.497399 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497158    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-envoy\") pod \"61c00532-bfa2-491b-80ff-25555a83cb2a\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") "
Jul 15 04:46:38.497399 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497251    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-credential-socket\") pod \"61c00532-bfa2-491b-80ff-25555a83cb2a\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") "
Jul 15 04:46:38.497399 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497276    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n55vd\" (UniqueName: \"kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-kube-api-access-n55vd\") pod \"61c00532-bfa2-491b-80ff-25555a83cb2a\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") "
Jul 15 04:46:38.497399 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497315    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c00532-bfa2-491b-80ff-25555a83cb2a-istiod-ca-cert\") pod \"61c00532-bfa2-491b-80ff-25555a83cb2a\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") "
Jul 15 04:46:38.497399 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497359    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-certs\") pod \"61c00532-bfa2-491b-80ff-25555a83cb2a\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") "
Jul 15 04:46:38.497399 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497389    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-token\") pod \"61c00532-bfa2-491b-80ff-25555a83cb2a\" (UID: \"61c00532-bfa2-491b-80ff-25555a83cb2a\") "
Jul 15 04:46:38.497836 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497395    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "61c00532-bfa2-491b-80ff-25555a83cb2a" (UID: "61c00532-bfa2-491b-80ff-25555a83cb2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:46:38.497836 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497746    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:38.497939 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497764    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61c00532-bfa2-491b-80ff-25555a83cb2a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "61c00532-bfa2-491b-80ff-25555a83cb2a" (UID: "61c00532-bfa2-491b-80ff-25555a83cb2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:46:38.497939 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497819    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "61c00532-bfa2-491b-80ff-25555a83cb2a" (UID: "61c00532-bfa2-491b-80ff-25555a83cb2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:46:38.498032 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.497940    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "61c00532-bfa2-491b-80ff-25555a83cb2a" (UID: "61c00532-bfa2-491b-80ff-25555a83cb2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:46:38.498151 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.498124    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-data" (OuterVolumeSpecName: "istio-data") pod "61c00532-bfa2-491b-80ff-25555a83cb2a" (UID: "61c00532-bfa2-491b-80ff-25555a83cb2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:46:38.500118 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.500085    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "61c00532-bfa2-491b-80ff-25555a83cb2a" (UID: "61c00532-bfa2-491b-80ff-25555a83cb2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:46:38.500245 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.500177    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-token" (OuterVolumeSpecName: "istio-token") pod "61c00532-bfa2-491b-80ff-25555a83cb2a" (UID: "61c00532-bfa2-491b-80ff-25555a83cb2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:46:38.500328 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.500304    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "61c00532-bfa2-491b-80ff-25555a83cb2a" (UID: "61c00532-bfa2-491b-80ff-25555a83cb2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:46:38.500395 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.500325    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-kube-api-access-n55vd" (OuterVolumeSpecName: "kube-api-access-n55vd") pod "61c00532-bfa2-491b-80ff-25555a83cb2a" (UID: "61c00532-bfa2-491b-80ff-25555a83cb2a"). InnerVolumeSpecName "kube-api-access-n55vd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:46:38.599081 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.599031    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:38.599081 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.599070    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n55vd\" (UniqueName: \"kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-kube-api-access-n55vd\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:38.599081 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.599083    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c00532-bfa2-491b-80ff-25555a83cb2a-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:38.599081 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.599092    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:38.599081 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.599101    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:38.599464 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.599109    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:38.599464 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.599116    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:38.599464 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.599123    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c00532-bfa2-491b-80ff-25555a83cb2a-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:46:38.863045 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.862939    2524 generic.go:356] "Generic (PLEG): container finished" podID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerID="ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f" exitCode=0
Jul 15 04:46:38.863045 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.863000    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"
Jul 15 04:46:38.863279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.863034    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" event={"ID":"61c00532-bfa2-491b-80ff-25555a83cb2a","Type":"ContainerDied","Data":"ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f"}
Jul 15 04:46:38.863279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.863093    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz" event={"ID":"61c00532-bfa2-491b-80ff-25555a83cb2a","Type":"ContainerDied","Data":"96face83662fd6172fc34fb04fbf8e596b1b1468192439b4b531ea3737452ce6"}
Jul 15 04:46:38.863279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.863126    2524 scope.go:117] "RemoveContainer" containerID="ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f"
Jul 15 04:46:38.864892 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.864866    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f" event={"ID":"fc388544-d033-4f50-a415-620b7e658220","Type":"ContainerStarted","Data":"3035e1d4944bec9de2ca0e3cf9829b2368670979a93a43ae593b4b293676d730"}
Jul 15 04:46:38.864977 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.864929    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:38.873954 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.873932    2524 scope.go:117] "RemoveContainer" containerID="c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c"
Jul 15 04:46:38.883136 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.883110    2524 scope.go:117] "RemoveContainer" containerID="ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f"
Jul 15 04:46:38.883484 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:46:38.883460    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f\": container with ID starting with ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f not found: ID does not exist" containerID="ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f"
Jul 15 04:46:38.883526 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.883494    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f"} err="failed to get container status \"ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f\": rpc error: code = NotFound desc = could not find container \"ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f\": container with ID starting with ac4266566518da762d7a3f87b0fd63eb4e368ba9828ff59359802849cd985a3f not found: ID does not exist"
Jul 15 04:46:38.883526 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.883512    2524 scope.go:117] "RemoveContainer" containerID="c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c"
Jul 15 04:46:38.883807 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:46:38.883788    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c\": container with ID starting with c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c not found: ID does not exist" containerID="c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c"
Jul 15 04:46:38.883885 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.883811    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c"} err="failed to get container status \"c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c\": rpc error: code = NotFound desc = could not find container \"c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c\": container with ID starting with c4b96cca9bad6dcd54744b9f3d16640616f5ba3cddbc48ee2917c7104bc7a19c not found: ID does not exist"
Jul 15 04:46:38.892938 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.892891    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f" podStartSLOduration=1.496512179 podStartE2EDuration="1.892877421s" totalImagesPullingTime="396.365242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:46:38.891835279 +0000 UTC m=+2428.203216527" watchObservedRunningTime="2026-07-15 04:46:38.892877421 +0000 UTC m=+2428.204258667"
Jul 15 04:46:38.904867 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.904833    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"]
Jul 15 04:46:38.908145 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:38.908115    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0135c8-openshift-default-c66bcb65c-kd6xz"]
Jul 15 04:46:39.433575 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:39.433543    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" path="/var/lib/kubelet/pods/61c00532-bfa2-491b-80ff-25555a83cb2a/volumes"
Jul 15 04:46:42.003374 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.003336    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"]
Jul 15 04:46:42.003951 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.003927    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy"
Jul 15 04:46:42.004094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.003955    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy"
Jul 15 04:46:42.004094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.004062    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy"
Jul 15 04:46:42.004094 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.004077    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy"
Jul 15 04:46:42.008898 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.008866    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.011614 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.011572    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3a7aa4-openshift-default-dockercfg-qjlnb\""
Jul 15 04:46:42.017853 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.017827    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"]
Jul 15 04:46:42.136080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.136037    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-credential-socket\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.136080 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.136083    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-envoy\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.136339 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.136109    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-token\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.136339 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.136214    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-socket\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.136339 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.136260    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hwbz\" (UniqueName: \"kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-kube-api-access-7hwbz\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.136339 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.136300    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-podinfo\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.136471 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.136350    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-data\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.136471 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.136414    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istiod-ca-cert\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.136533 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.136468    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-certs\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237003 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.236967    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-credential-socket\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237003 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237007    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-envoy\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237237 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237176    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-token\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237237 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237217    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-socket\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237342 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237257    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hwbz\" (UniqueName: \"kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-kube-api-access-7hwbz\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237342 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237312    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-podinfo\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237436 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237344    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-credential-socket\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237436 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237366    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-data\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237436 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237420    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istiod-ca-cert\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237583 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237509    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-certs\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237583 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237520    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-socket\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237765 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237744    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-data\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.237816 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.237780    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-certs\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.238143 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.238079    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istiod-ca-cert\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.239749 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.239718    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-envoy\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.240332 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.240306    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-podinfo\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.246035 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.246010    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-token\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.246216 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.246194    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hwbz\" (UniqueName: \"kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-kube-api-access-7hwbz\") pod \"e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") " pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.321449 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.321352    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:42.477496 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.477464    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"]
Jul 15 04:46:42.480091 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:46:42.480061    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2638c5eb_f45d_4d1e_9014_b61dd1a17b02.slice/crio-06da4e83cc4c25127c10e768dcf6d2a48986bf30e070f156d74aa1f6d936fc63 WatchSource:0}: Error finding container 06da4e83cc4c25127c10e768dcf6d2a48986bf30e070f156d74aa1f6d936fc63: Status 404 returned error can't find the container with id 06da4e83cc4c25127c10e768dcf6d2a48986bf30e070f156d74aa1f6d936fc63
Jul 15 04:46:42.482526 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.482488    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:42.482625 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.482567    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:42.482625 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.482616    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:42.890160 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.890116    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" event={"ID":"2638c5eb-f45d-4d1e-9014-b61dd1a17b02","Type":"ContainerStarted","Data":"73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68"}
Jul 15 04:46:42.890359 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.890184    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" event={"ID":"2638c5eb-f45d-4d1e-9014-b61dd1a17b02","Type":"ContainerStarted","Data":"06da4e83cc4c25127c10e768dcf6d2a48986bf30e070f156d74aa1f6d936fc63"}
Jul 15 04:46:42.911552 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:42.911495    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" podStartSLOduration=1.911476994 podStartE2EDuration="1.911476994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:46:42.909389304 +0000 UTC m=+2432.220770552" watchObservedRunningTime="2026-07-15 04:46:42.911476994 +0000 UTC m=+2432.222858240"
Jul 15 04:46:43.322176 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:43.322073    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:43.328532 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:43.328505    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:43.895117 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:43.895086    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:43.896254 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:43.896231    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:44.885466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:44.885429    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:46:45.251819 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.251715    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"]
Jul 15 04:46:45.252349 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.252332    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy"
Jul 15 04:46:45.252398 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.252354    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="61c00532-bfa2-491b-80ff-25555a83cb2a" containerName="istio-proxy"
Jul 15 04:46:45.256141 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.256118    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:45.264511 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.262942    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-3a7aa4-serving-cert\""
Jul 15 04:46:45.277631 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.277317    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/766da2c2-8748-4a81-8c53-1f9e019d8bc4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:45.277631 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.277404    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd642\" (UniqueName: \"kubernetes.io/projected/766da2c2-8748-4a81-8c53-1f9e019d8bc4-kube-api-access-jd642\") pod \"maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:45.277631 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.277437    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/766da2c2-8748-4a81-8c53-1f9e019d8bc4-maas-api-tls\") pod \"maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:45.278904 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.278868    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"]
Jul 15 04:46:45.380962 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.378451    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd642\" (UniqueName: \"kubernetes.io/projected/766da2c2-8748-4a81-8c53-1f9e019d8bc4-kube-api-access-jd642\") pod \"maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:45.380962 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.378527    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/766da2c2-8748-4a81-8c53-1f9e019d8bc4-maas-api-tls\") pod \"maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:45.380962 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.378730    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/766da2c2-8748-4a81-8c53-1f9e019d8bc4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:45.380962 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.379897    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/766da2c2-8748-4a81-8c53-1f9e019d8bc4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:45.392907 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.392818    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/766da2c2-8748-4a81-8c53-1f9e019d8bc4-maas-api-tls\") pod \"maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:45.394626 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.394545    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd642\" (UniqueName: \"kubernetes.io/projected/766da2c2-8748-4a81-8c53-1f9e019d8bc4-kube-api-access-jd642\") pod \"maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:45.591119 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.590999    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:45.784922 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.784291    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"]
Jul 15 04:46:45.788410 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:46:45.788372    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod766da2c2_8748_4a81_8c53_1f9e019d8bc4.slice/crio-39cd77086b688f30f97ac00fb4143a276f613e5c0c4daeb532b8e6c101e7365c WatchSource:0}: Error finding container 39cd77086b688f30f97ac00fb4143a276f613e5c0c4daeb532b8e6c101e7365c: Status 404 returned error can't find the container with id 39cd77086b688f30f97ac00fb4143a276f613e5c0c4daeb532b8e6c101e7365c
Jul 15 04:46:45.906247 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:45.906210    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc" event={"ID":"766da2c2-8748-4a81-8c53-1f9e019d8bc4","Type":"ContainerStarted","Data":"39cd77086b688f30f97ac00fb4143a276f613e5c0c4daeb532b8e6c101e7365c"}
Jul 15 04:46:46.912989 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:46.912947    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc" event={"ID":"766da2c2-8748-4a81-8c53-1f9e019d8bc4","Type":"ContainerStarted","Data":"41046f3f2880d1d355ec02ae3aeace213cfbe750581823b1b7a50252a836ad6c"}
Jul 15 04:46:46.913393 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:46.913050    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:46.945765 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:46.945712    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc" podStartSLOduration=1.664643711 podStartE2EDuration="1.945692706s" totalImagesPullingTime="281.048995ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:46:46.94384905 +0000 UTC m=+2436.255230332" watchObservedRunningTime="2026-07-15 04:46:46.945692706 +0000 UTC m=+2436.257073954"
Jul 15 04:46:50.955971 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:50.954664    2524 patch_prober.go:28] interesting pod/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": read tcp 10.133.0.2:37928->10.133.0.61:15021: read: connection reset by peer" start-of-body=
Jul 15 04:46:50.955971 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:50.954730    2524 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" podUID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": read tcp 10.133.0.2:37928->10.133.0.61:15021: read: connection reset by peer"
Jul 15 04:46:51.940517 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:51.940474    2524 generic.go:356] "Generic (PLEG): container finished" podID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerID="c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe" exitCode=0
Jul 15 04:46:51.940722 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:51.940547    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" event={"ID":"528f9441-66ec-4cc0-9245-0fb56a98d014","Type":"ContainerDied","Data":"c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe"}
Jul 15 04:46:51.941059 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:51.941042    2524 scope.go:117] "RemoveContainer" containerID="c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe"
Jul 15 04:46:51.942131 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:51.942099    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:51.942233 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:51.942168    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:51.942233 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:51.942201    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:52.924195 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:52.924164    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:46:52.950509 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:52.950467    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" event={"ID":"528f9441-66ec-4cc0-9245-0fb56a98d014","Type":"ContainerStarted","Data":"d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6"}
Jul 15 04:46:53.386994 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:53.386898    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:53.387485 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:53.387437    2524 patch_prober.go:28] interesting pod/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 15 04:46:53.387640 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:53.387528    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" podUID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 15 04:46:54.386769 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:54.386734    2524 patch_prober.go:28] interesting pod/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 15 04:46:54.387177 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:54.386813    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" podUID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 15 04:46:55.387240 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:55.387193    2524 patch_prober.go:28] interesting pod/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 15 04:46:55.387703 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:55.387273    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" podUID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 15 04:46:56.387912 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:56.387865    2524 patch_prober.go:28] interesting pod/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 15 04:46:56.388413 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:56.387936    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" podUID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 15 04:46:57.391048 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:57.390970    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:57.391464 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:57.391256    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:57.392097 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:57.392079    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:46:57.984498 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:57.984461    2524 generic.go:356] "Generic (PLEG): container finished" podID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerID="73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68" exitCode=0
Jul 15 04:46:57.984755 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:57.984541    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" event={"ID":"2638c5eb-f45d-4d1e-9014-b61dd1a17b02","Type":"ContainerDied","Data":"73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68"}
Jul 15 04:46:57.985113 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:57.985093    2524 scope.go:117] "RemoveContainer" containerID="73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68"
Jul 15 04:46:57.986230 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:57.986195    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:57.986318 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:57.986290    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:57.986364 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:57.986345    2524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 15 04:46:58.895100 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:58.895064    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:59.000048 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:59.000003    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" event={"ID":"2638c5eb-f45d-4d1e-9014-b61dd1a17b02","Type":"ContainerStarted","Data":"4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808"}
Jul 15 04:46:59.322505 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:59.322408    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:46:59.323907 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:59.323878    2524 patch_prober.go:28] interesting pod/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 15 04:46:59.324036 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:46:59.323933    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 15 04:47:00.322500 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:47:00.322458    2524 patch_prober.go:28] interesting pod/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 15 04:47:00.322975 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:47:00.322523    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 15 04:47:01.322063 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:47:01.322018    2524 patch_prober.go:28] interesting pod/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 15 04:47:01.322282 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:47:01.322082    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 15 04:47:02.321775 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:47:02.321735    2524 patch_prober.go:28] interesting pod/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 15 04:47:02.322211 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:47:02.321807    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 15 04:47:03.322010 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:47:03.321965    2524 patch_prober.go:28] interesting pod/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 15 04:47:03.322430 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:47:03.322035    2524 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 15 04:47:04.326935 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:47:04.326900    2524 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:47:04.327425 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:47:04.327309    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:47:04.328200 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:47:04.328178    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:51:11.514279 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:51:11.514207    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:51:11.519404 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:51:11.519380    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kmwrc_f853b65c-ad5d-4187-b60b-bdb5715a1873/ovn-acl-logging/0.log"
Jul 15 04:52:42.586217 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:42.586182    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"]
Jul 15 04:52:42.586783 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:42.586453    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f" podUID="fc388544-d033-4f50-a415-620b7e658220" containerName="maas-api" containerID="cri-o://3035e1d4944bec9de2ca0e3cf9829b2368670979a93a43ae593b4b293676d730" gracePeriod=30
Jul 15 04:52:42.739928 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:42.739834    2524 generic.go:356] "Generic (PLEG): container finished" podID="fc388544-d033-4f50-a415-620b7e658220" containerID="3035e1d4944bec9de2ca0e3cf9829b2368670979a93a43ae593b4b293676d730" exitCode=0
Jul 15 04:52:42.739928 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:42.739879    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f" event={"ID":"fc388544-d033-4f50-a415-620b7e658220","Type":"ContainerDied","Data":"3035e1d4944bec9de2ca0e3cf9829b2368670979a93a43ae593b4b293676d730"}
Jul 15 04:52:42.854614 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:42.854577    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:52:42.899395 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:42.899365    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb6bj\" (UniqueName: \"kubernetes.io/projected/fc388544-d033-4f50-a415-620b7e658220-kube-api-access-hb6bj\") pod \"fc388544-d033-4f50-a415-620b7e658220\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") "
Jul 15 04:52:42.899568 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:42.899443    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc388544-d033-4f50-a415-620b7e658220-openshift-service-ca\") pod \"fc388544-d033-4f50-a415-620b7e658220\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") "
Jul 15 04:52:42.899568 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:42.899521    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc388544-d033-4f50-a415-620b7e658220-maas-api-tls\") pod \"fc388544-d033-4f50-a415-620b7e658220\" (UID: \"fc388544-d033-4f50-a415-620b7e658220\") "
Jul 15 04:52:42.899940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:42.899907    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc388544-d033-4f50-a415-620b7e658220-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fc388544-d033-4f50-a415-620b7e658220" (UID: "fc388544-d033-4f50-a415-620b7e658220"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:52:42.901823 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:42.901800    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc388544-d033-4f50-a415-620b7e658220-kube-api-access-hb6bj" (OuterVolumeSpecName: "kube-api-access-hb6bj") pod "fc388544-d033-4f50-a415-620b7e658220" (UID: "fc388544-d033-4f50-a415-620b7e658220"). InnerVolumeSpecName "kube-api-access-hb6bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:52:42.901823 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:42.901799    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc388544-d033-4f50-a415-620b7e658220-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fc388544-d033-4f50-a415-620b7e658220" (UID: "fc388544-d033-4f50-a415-620b7e658220"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:52:43.000635 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:43.000553    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc388544-d033-4f50-a415-620b7e658220-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:52:43.000635 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:43.000585    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc388544-d033-4f50-a415-620b7e658220-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:52:43.000635 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:43.000612    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb6bj\" (UniqueName: \"kubernetes.io/projected/fc388544-d033-4f50-a415-620b7e658220-kube-api-access-hb6bj\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:52:43.746153 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:43.746070    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"
Jul 15 04:52:43.746153 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:43.746082    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f" event={"ID":"fc388544-d033-4f50-a415-620b7e658220","Type":"ContainerDied","Data":"61cd3eb8882950f511c2d8384ffe735b6b5be482e65c4a725f22a756156b60f2"}
Jul 15 04:52:43.746153 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:43.746154    2524 scope.go:117] "RemoveContainer" containerID="3035e1d4944bec9de2ca0e3cf9829b2368670979a93a43ae593b4b293676d730"
Jul 15 04:52:43.765990 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:43.765957    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"]
Jul 15 04:52:43.768003 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:43.767980    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c2a8a9-9f97bfb96-lct9f"]
Jul 15 04:52:45.432788 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:45.432740    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc388544-d033-4f50-a415-620b7e658220" path="/var/lib/kubelet/pods/fc388544-d033-4f50-a415-620b7e658220/volumes"
Jul 15 04:52:57.748505 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:57.748458    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"]
Jul 15 04:52:57.749019 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:52:57.748761    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" podUID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerName="istio-proxy" containerID="cri-o://d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6" gracePeriod=30
Jul 15 04:53:03.012940 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.012871    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:53:03.096875 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.096839    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-socket\") pod \"528f9441-66ec-4cc0-9245-0fb56a98d014\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") "
Jul 15 04:53:03.097086 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.096886    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-credential-socket\") pod \"528f9441-66ec-4cc0-9245-0fb56a98d014\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") "
Jul 15 04:53:03.097086 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.096912    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-data\") pod \"528f9441-66ec-4cc0-9245-0fb56a98d014\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") "
Jul 15 04:53:03.097086 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.096963    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-certs\") pod \"528f9441-66ec-4cc0-9245-0fb56a98d014\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") "
Jul 15 04:53:03.097086 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.096991    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/528f9441-66ec-4cc0-9245-0fb56a98d014-istiod-ca-cert\") pod \"528f9441-66ec-4cc0-9245-0fb56a98d014\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") "
Jul 15 04:53:03.097086 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097050    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-podinfo\") pod \"528f9441-66ec-4cc0-9245-0fb56a98d014\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") "
Jul 15 04:53:03.097367 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097091    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9djd6\" (UniqueName: \"kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-kube-api-access-9djd6\") pod \"528f9441-66ec-4cc0-9245-0fb56a98d014\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") "
Jul 15 04:53:03.097367 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097118    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-envoy\") pod \"528f9441-66ec-4cc0-9245-0fb56a98d014\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") "
Jul 15 04:53:03.097367 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097218    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "528f9441-66ec-4cc0-9245-0fb56a98d014" (UID: "528f9441-66ec-4cc0-9245-0fb56a98d014"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:53:03.097367 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097308    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-token\") pod \"528f9441-66ec-4cc0-9245-0fb56a98d014\" (UID: \"528f9441-66ec-4cc0-9245-0fb56a98d014\") "
Jul 15 04:53:03.097554 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097237    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "528f9441-66ec-4cc0-9245-0fb56a98d014" (UID: "528f9441-66ec-4cc0-9245-0fb56a98d014"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:53:03.097554 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097364    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "528f9441-66ec-4cc0-9245-0fb56a98d014" (UID: "528f9441-66ec-4cc0-9245-0fb56a98d014"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:53:03.097554 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097483    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/528f9441-66ec-4cc0-9245-0fb56a98d014-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "528f9441-66ec-4cc0-9245-0fb56a98d014" (UID: "528f9441-66ec-4cc0-9245-0fb56a98d014"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:53:03.097797 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097777    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/528f9441-66ec-4cc0-9245-0fb56a98d014-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:03.097862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097799    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:03.097862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097808    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:03.097862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.097817    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:03.098332 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.098303    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-data" (OuterVolumeSpecName: "istio-data") pod "528f9441-66ec-4cc0-9245-0fb56a98d014" (UID: "528f9441-66ec-4cc0-9245-0fb56a98d014"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:53:03.099670 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.099640    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-token" (OuterVolumeSpecName: "istio-token") pod "528f9441-66ec-4cc0-9245-0fb56a98d014" (UID: "528f9441-66ec-4cc0-9245-0fb56a98d014"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:53:03.099791 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.099701    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "528f9441-66ec-4cc0-9245-0fb56a98d014" (UID: "528f9441-66ec-4cc0-9245-0fb56a98d014"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:53:03.099896 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.099876    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "528f9441-66ec-4cc0-9245-0fb56a98d014" (UID: "528f9441-66ec-4cc0-9245-0fb56a98d014"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:53:03.099954 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.099916    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-kube-api-access-9djd6" (OuterVolumeSpecName: "kube-api-access-9djd6") pod "528f9441-66ec-4cc0-9245-0fb56a98d014" (UID: "528f9441-66ec-4cc0-9245-0fb56a98d014"). InnerVolumeSpecName "kube-api-access-9djd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:53:03.198317 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.198271    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:03.198317 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.198309    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9djd6\" (UniqueName: \"kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-kube-api-access-9djd6\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:03.198317 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.198319    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:03.198317 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.198328    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:03.198633 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.198336    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/528f9441-66ec-4cc0-9245-0fb56a98d014-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:03.847333 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.847243    2524 generic.go:356] "Generic (PLEG): container finished" podID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerID="d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6" exitCode=0
Jul 15 04:53:03.847333 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.847285    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"
Jul 15 04:53:03.847520 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.847334    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" event={"ID":"528f9441-66ec-4cc0-9245-0fb56a98d014","Type":"ContainerDied","Data":"d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6"}
Jul 15 04:53:03.847520 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.847392    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk" event={"ID":"528f9441-66ec-4cc0-9245-0fb56a98d014","Type":"ContainerDied","Data":"c59ca656498bb0d95f279faf6fac005a869f65cbb3a1c98ec06a826fd4845ba6"}
Jul 15 04:53:03.847520 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.847424    2524 scope.go:117] "RemoveContainer" containerID="d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6"
Jul 15 04:53:03.856226 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.856206    2524 scope.go:117] "RemoveContainer" containerID="c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe"
Jul 15 04:53:03.865139 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.865117    2524 scope.go:117] "RemoveContainer" containerID="d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6"
Jul 15 04:53:03.865441 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:53:03.865421    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6\": container with ID starting with d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6 not found: ID does not exist" containerID="d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6"
Jul 15 04:53:03.865500 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.865451    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6"} err="failed to get container status \"d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6\": rpc error: code = NotFound desc = could not find container \"d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6\": container with ID starting with d8ee89f6d5fd3ba7fac60b10721c56b22613f875f4223e24db4913f73dbcdcb6 not found: ID does not exist"
Jul 15 04:53:03.865500 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.865470    2524 scope.go:117] "RemoveContainer" containerID="c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe"
Jul 15 04:53:03.865761 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.865735    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"]
Jul 15 04:53:03.865761 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:53:03.865750    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe\": container with ID starting with c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe not found: ID does not exist" containerID="c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe"
Jul 15 04:53:03.865900 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.865775    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe"} err="failed to get container status \"c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe\": rpc error: code = NotFound desc = could not find container \"c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe\": container with ID starting with c95e0d9b45ec6678f37d21133f4690ced19503af36710b847e4ab6ebcb0b42fe not found: ID does not exist"
Jul 15 04:53:03.872182 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:03.872158    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-c2a8a9-openshift-default-588566d7ff-z9pmk"]
Jul 15 04:53:05.435034 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:05.435002    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="528f9441-66ec-4cc0-9245-0fb56a98d014" path="/var/lib/kubelet/pods/528f9441-66ec-4cc0-9245-0fb56a98d014/volumes"
Jul 15 04:53:08.724976 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:08.724941    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"]
Jul 15 04:53:08.725466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:08.725152    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc" podUID="766da2c2-8748-4a81-8c53-1f9e019d8bc4" containerName="maas-api" containerID="cri-o://41046f3f2880d1d355ec02ae3aeace213cfbe750581823b1b7a50252a836ad6c" gracePeriod=30
Jul 15 04:53:08.881100 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:08.881063    2524 generic.go:356] "Generic (PLEG): container finished" podID="766da2c2-8748-4a81-8c53-1f9e019d8bc4" containerID="41046f3f2880d1d355ec02ae3aeace213cfbe750581823b1b7a50252a836ad6c" exitCode=0
Jul 15 04:53:08.881277 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:08.881115    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc" event={"ID":"766da2c2-8748-4a81-8c53-1f9e019d8bc4","Type":"ContainerDied","Data":"41046f3f2880d1d355ec02ae3aeace213cfbe750581823b1b7a50252a836ad6c"}
Jul 15 04:53:08.980983 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:08.980924    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:53:09.158470 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.158435    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd642\" (UniqueName: \"kubernetes.io/projected/766da2c2-8748-4a81-8c53-1f9e019d8bc4-kube-api-access-jd642\") pod \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") "
Jul 15 04:53:09.158692 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.158533    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/766da2c2-8748-4a81-8c53-1f9e019d8bc4-maas-api-tls\") pod \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") "
Jul 15 04:53:09.158692 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.158620    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/766da2c2-8748-4a81-8c53-1f9e019d8bc4-openshift-service-ca\") pod \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\" (UID: \"766da2c2-8748-4a81-8c53-1f9e019d8bc4\") "
Jul 15 04:53:09.158986 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.158962    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/766da2c2-8748-4a81-8c53-1f9e019d8bc4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "766da2c2-8748-4a81-8c53-1f9e019d8bc4" (UID: "766da2c2-8748-4a81-8c53-1f9e019d8bc4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:53:09.160745 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.160720    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/766da2c2-8748-4a81-8c53-1f9e019d8bc4-kube-api-access-jd642" (OuterVolumeSpecName: "kube-api-access-jd642") pod "766da2c2-8748-4a81-8c53-1f9e019d8bc4" (UID: "766da2c2-8748-4a81-8c53-1f9e019d8bc4"). InnerVolumeSpecName "kube-api-access-jd642". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:53:09.160846 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.160813    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/766da2c2-8748-4a81-8c53-1f9e019d8bc4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "766da2c2-8748-4a81-8c53-1f9e019d8bc4" (UID: "766da2c2-8748-4a81-8c53-1f9e019d8bc4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 04:53:09.259969 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.259866    2524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/766da2c2-8748-4a81-8c53-1f9e019d8bc4-maas-api-tls\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:09.259969 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.259909    2524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/766da2c2-8748-4a81-8c53-1f9e019d8bc4-openshift-service-ca\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:09.259969 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.259923    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd642\" (UniqueName: \"kubernetes.io/projected/766da2c2-8748-4a81-8c53-1f9e019d8bc4-kube-api-access-jd642\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:09.887554 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.887521    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"
Jul 15 04:53:09.887554 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.887552    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc" event={"ID":"766da2c2-8748-4a81-8c53-1f9e019d8bc4","Type":"ContainerDied","Data":"39cd77086b688f30f97ac00fb4143a276f613e5c0c4daeb532b8e6c101e7365c"}
Jul 15 04:53:09.888144 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.887640    2524 scope.go:117] "RemoveContainer" containerID="41046f3f2880d1d355ec02ae3aeace213cfbe750581823b1b7a50252a836ad6c"
Jul 15 04:53:09.903902 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.903876    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"]
Jul 15 04:53:09.910255 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:09.910234    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3a7aa4-77fbf9f688-46tgc"]
Jul 15 04:53:11.432655 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:11.432618    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="766da2c2-8748-4a81-8c53-1f9e019d8bc4" path="/var/lib/kubelet/pods/766da2c2-8748-4a81-8c53-1f9e019d8bc4/volumes"
Jul 15 04:53:23.939562 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:23.939528    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"]
Jul 15 04:53:23.940043 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:23.939790    2524 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy" containerID="cri-o://4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808" gracePeriod=30
Jul 15 04:53:29.181692 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.181667    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:53:29.244001 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.243921    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-credential-socket\") pod \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") "
Jul 15 04:53:29.244001 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.243975    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-envoy\") pod \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") "
Jul 15 04:53:29.244001 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.243998    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-socket\") pod \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") "
Jul 15 04:53:29.244217 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244061    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-data\") pod \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") "
Jul 15 04:53:29.244217 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244104    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-token\") pod \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") "
Jul 15 04:53:29.244217 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244107    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2638c5eb-f45d-4d1e-9014-b61dd1a17b02" (UID: "2638c5eb-f45d-4d1e-9014-b61dd1a17b02"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:53:29.244217 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244145    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-podinfo\") pod \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") "
Jul 15 04:53:29.244217 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244204    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-certs\") pod \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") "
Jul 15 04:53:29.244472 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244247    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hwbz\" (UniqueName: \"kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-kube-api-access-7hwbz\") pod \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") "
Jul 15 04:53:29.244472 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244288    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2638c5eb-f45d-4d1e-9014-b61dd1a17b02" (UID: "2638c5eb-f45d-4d1e-9014-b61dd1a17b02"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:53:29.244472 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244348    2524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istiod-ca-cert\") pod \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\" (UID: \"2638c5eb-f45d-4d1e-9014-b61dd1a17b02\") "
Jul 15 04:53:29.244836 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244718    2524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-credential-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:29.244836 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244743    2524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-socket\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:29.244836 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244775    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2638c5eb-f45d-4d1e-9014-b61dd1a17b02" (UID: "2638c5eb-f45d-4d1e-9014-b61dd1a17b02"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 04:53:29.245223 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.244849    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2638c5eb-f45d-4d1e-9014-b61dd1a17b02" (UID: "2638c5eb-f45d-4d1e-9014-b61dd1a17b02"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:53:29.245223 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.245071    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-data" (OuterVolumeSpecName: "istio-data") pod "2638c5eb-f45d-4d1e-9014-b61dd1a17b02" (UID: "2638c5eb-f45d-4d1e-9014-b61dd1a17b02"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:53:29.246465 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.246439    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-token" (OuterVolumeSpecName: "istio-token") pod "2638c5eb-f45d-4d1e-9014-b61dd1a17b02" (UID: "2638c5eb-f45d-4d1e-9014-b61dd1a17b02"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:53:29.246576 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.246464    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2638c5eb-f45d-4d1e-9014-b61dd1a17b02" (UID: "2638c5eb-f45d-4d1e-9014-b61dd1a17b02"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 04:53:29.246862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.246842    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2638c5eb-f45d-4d1e-9014-b61dd1a17b02" (UID: "2638c5eb-f45d-4d1e-9014-b61dd1a17b02"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 04:53:29.246862 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.246853    2524 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-kube-api-access-7hwbz" (OuterVolumeSpecName: "kube-api-access-7hwbz") pod "2638c5eb-f45d-4d1e-9014-b61dd1a17b02" (UID: "2638c5eb-f45d-4d1e-9014-b61dd1a17b02"). InnerVolumeSpecName "kube-api-access-7hwbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 04:53:29.345265 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.345235    2524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hwbz\" (UniqueName: \"kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-kube-api-access-7hwbz\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:29.345265 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.345263    2524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istiod-ca-cert\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:29.345466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.345275    2524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-envoy\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:29.345466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.345287    2524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-data\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:29.345466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.345297    2524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-token\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:29.345466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.345307    2524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-istio-podinfo\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:29.345466 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.345318    2524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2638c5eb-f45d-4d1e-9014-b61dd1a17b02-workload-certs\") on node \"ip-10-0-129-76.ec2.internal\" DevicePath \"\""
Jul 15 04:53:29.980915 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.980882    2524 generic.go:356] "Generic (PLEG): container finished" podID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerID="4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808" exitCode=0
Jul 15 04:53:29.981116 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.980931    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" event={"ID":"2638c5eb-f45d-4d1e-9014-b61dd1a17b02","Type":"ContainerDied","Data":"4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808"}
Jul 15 04:53:29.981116 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.980936    2524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"
Jul 15 04:53:29.981116 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.980974    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459" event={"ID":"2638c5eb-f45d-4d1e-9014-b61dd1a17b02","Type":"ContainerDied","Data":"06da4e83cc4c25127c10e768dcf6d2a48986bf30e070f156d74aa1f6d936fc63"}
Jul 15 04:53:29.981116 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.980996    2524 scope.go:117] "RemoveContainer" containerID="4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808"
Jul 15 04:53:29.991879 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:29.991833    2524 scope.go:117] "RemoveContainer" containerID="73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68"
Jul 15 04:53:30.000307 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:30.000281    2524 scope.go:117] "RemoveContainer" containerID="4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808"
Jul 15 04:53:30.000640 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:53:30.000589    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808\": container with ID starting with 4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808 not found: ID does not exist" containerID="4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808"
Jul 15 04:53:30.000761 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:30.000650    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808"} err="failed to get container status \"4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808\": rpc error: code = NotFound desc = could not find container \"4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808\": container with ID starting with 4995d604b1430a83de0fee5549b05f25aca720ee18fbf0b2eddfffbc23615808 not found: ID does not exist"
Jul 15 04:53:30.000761 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:30.000669    2524 scope.go:117] "RemoveContainer" containerID="73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68"
Jul 15 04:53:30.000962 ip-10-0-129-76 kubenswrapper[2524]: E0715 04:53:30.000943    2524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68\": container with ID starting with 73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68 not found: ID does not exist" containerID="73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68"
Jul 15 04:53:30.001008 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:30.000970    2524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68"} err="failed to get container status \"73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68\": rpc error: code = NotFound desc = could not find container \"73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68\": container with ID starting with 73d6870bd3f81a35a0dc0355b9134ae0ccf3d291e28364a39420ee78e805ac68 not found: ID does not exist"
Jul 15 04:53:30.002166 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:30.002146    2524 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"]
Jul 15 04:53:30.006738 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:30.006715    2524 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3a7aa4-openshift-default-6cfbcfc79-mn459"]
Jul 15 04:53:31.433237 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:31.433201    2524 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" path="/var/lib/kubelet/pods/2638c5eb-f45d-4d1e-9014-b61dd1a17b02/volumes"
Jul 15 04:53:33.941687 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:33.941632    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67d859fc9c-kh8zv_aa2b64d7-833d-453c-bd68-0182c97e141d/manager/0.log"
Jul 15 04:53:34.461315 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:34.461285    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddd85b7f-dl9pg_863afdaf-88d5-467f-aaa4-c0a1af6bbfe8/copy-manifests/0.log"
Jul 15 04:53:34.467973 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:34.467946    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddd85b7f-dl9pg_863afdaf-88d5-467f-aaa4-c0a1af6bbfe8/manager/0.log"
Jul 15 04:53:34.969033 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:34.969006    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-664cd857c8-qv8rg_657ae513-60d6-471a-8bb6-7d2b2e27dd32/maas-api/0.log"
Jul 15 04:53:35.085678 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:35.085637    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734830-g48db_2ce7bbf0-418f-4356-8563-51f73b2e9441/cleanup/0.log"
Jul 15 04:53:35.203558 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:35.203527    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734845-9k98d_494a8ea8-2eee-42b2-b649-a9ebd283dd8a/cleanup/0.log"
Jul 15 04:53:36.689438 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:36.689410    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-m8pr4_33f2aa1e-a28c-41aa-9671-780931d50bbe/postgres/0.log"
Jul 15 04:53:39.394345 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:39.394318    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-95pfs_b825790a-c79c-472d-9c56-ac3a248491bc/discovery/0.log"
Jul 15 04:53:39.877511 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:39.877479    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-7b9nq_adefd0a4-cd6b-4b91-9aa9-da45df2ca089/payload-pre-processing/0.log"
Jul 15 04:53:40.000802 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:40.000770    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-wrt6d_809162ae-0b69-4f2d-ba1a-7a97ec49f18c/payload-processing/0.log"
Jul 15 04:53:40.721053 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:40.721029    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-slhfq_10a6c75c-71fb-4b4d-9987-b89e2c511e9a/main/0.log"
Jul 15 04:53:48.440781 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:48.440751    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-swf7w_bb2bdcea-aedf-4f70-8aae-b2901b9c4a83/global-pull-secret-syncer/0.log"
Jul 15 04:53:48.527425 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:48.527398    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-d9v7v_58f571a6-f29b-4885-bac5-8e637b7a2027/konnectivity-agent/0.log"
Jul 15 04:53:48.626136 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:48.626102    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-76.ec2.internal_f15257a2672d183728dc10e769d94d89/haproxy/0.log"
Jul 15 04:53:56.632890 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:56.632860    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95rx6_25949e1b-9c1a-4126-a277-f73daa94bbe7/node-exporter/0.log"
Jul 15 04:53:56.654701 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:56.654678    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95rx6_25949e1b-9c1a-4126-a277-f73daa94bbe7/kube-rbac-proxy/0.log"
Jul 15 04:53:56.676025 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:56.676002    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95rx6_25949e1b-9c1a-4126-a277-f73daa94bbe7/init-textfile/0.log"
Jul 15 04:53:56.873234 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:56.873197    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8/prometheus/0.log"
Jul 15 04:53:56.893642 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:56.893548    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8/config-reloader/0.log"
Jul 15 04:53:56.913603 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:56.913568    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8/thanos-sidecar/0.log"
Jul 15 04:53:56.933355 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:56.933333    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8/kube-rbac-proxy-web/0.log"
Jul 15 04:53:56.955743 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:56.955723    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8/kube-rbac-proxy/0.log"
Jul 15 04:53:56.988713 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:56.988685    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8/kube-rbac-proxy-thanos/0.log"
Jul 15 04:53:57.011991 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.011967    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16ef1fc5-bfa6-473a-8ccf-e5449ac1d7e8/init-config-reloader/0.log"
Jul 15 04:53:57.037495 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.037465    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-6mt6f_3c182da7-587e-40d9-b1db-9594a1ef2569/prometheus-operator/0.log"
Jul 15 04:53:57.055723 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.055697    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-6mt6f_3c182da7-587e-40d9-b1db-9594a1ef2569/kube-rbac-proxy/0.log"
Jul 15 04:53:57.079114 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.079086    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-ptjnk_a8e6a017-a7a6-4bd3-bd99-0043a1c28cd0/prometheus-operator-admission-webhook/0.log"
Jul 15 04:53:57.182439 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.182359    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7667bdb9fc-4xpzj_603aa743-0412-4183-a7e1-17ceebf776a9/thanos-query/0.log"
Jul 15 04:53:57.202703 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.202655    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7667bdb9fc-4xpzj_603aa743-0412-4183-a7e1-17ceebf776a9/kube-rbac-proxy-web/0.log"
Jul 15 04:53:57.222320 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.222300    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7667bdb9fc-4xpzj_603aa743-0412-4183-a7e1-17ceebf776a9/kube-rbac-proxy/0.log"
Jul 15 04:53:57.243735 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.243712    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7667bdb9fc-4xpzj_603aa743-0412-4183-a7e1-17ceebf776a9/prom-label-proxy/0.log"
Jul 15 04:53:57.264431 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.264414    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7667bdb9fc-4xpzj_603aa743-0412-4183-a7e1-17ceebf776a9/kube-rbac-proxy-rules/0.log"
Jul 15 04:53:57.292770 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.292748    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7667bdb9fc-4xpzj_603aa743-0412-4183-a7e1-17ceebf776a9/kube-rbac-proxy-metrics/0.log"
Jul 15 04:53:57.954827 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.954782    2524 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"]
Jul 15 04:53:57.955240 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955182    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy"
Jul 15 04:53:57.955240 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955200    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy"
Jul 15 04:53:57.955240 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955211    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerName="istio-proxy"
Jul 15 04:53:57.955240 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955220    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerName="istio-proxy"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955245    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="766da2c2-8748-4a81-8c53-1f9e019d8bc4" containerName="maas-api"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955254    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="766da2c2-8748-4a81-8c53-1f9e019d8bc4" containerName="maas-api"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955268    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955275    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955293    2524 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc388544-d033-4f50-a415-620b7e658220" containerName="maas-api"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955300    2524 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc388544-d033-4f50-a415-620b7e658220" containerName="maas-api"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955401    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerName="istio-proxy"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955414    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955422    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="766da2c2-8748-4a81-8c53-1f9e019d8bc4" containerName="maas-api"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955430    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="528f9441-66ec-4cc0-9245-0fb56a98d014" containerName="istio-proxy"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955440    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="2638c5eb-f45d-4d1e-9014-b61dd1a17b02" containerName="istio-proxy"
Jul 15 04:53:57.955457 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.955450    2524 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc388544-d033-4f50-a415-620b7e658220" containerName="maas-api"
Jul 15 04:53:57.958482 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.958466    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:57.960834 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.960813    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lg2gg\"/\"openshift-service-ca.crt\""
Jul 15 04:53:57.960963 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.960817    2524 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lg2gg\"/\"default-dockercfg-j6wft\""
Jul 15 04:53:57.961713 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.961697    2524 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lg2gg\"/\"kube-root-ca.crt\""
Jul 15 04:53:57.967575 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:57.967553    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"]
Jul 15 04:53:58.099997 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.099958    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-proc\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.100184 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.100005    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss2tt\" (UniqueName: \"kubernetes.io/projected/d611de83-aa6f-401d-964c-6b87522cab23-kube-api-access-ss2tt\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.100184 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.100066    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-sys\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.100184 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.100138    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-podres\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.100345 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.100233    2524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-lib-modules\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.201381 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.201346    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-podres\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.201624 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.201406    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-lib-modules\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.201624 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.201436    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-proc\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.201624 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.201464    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss2tt\" (UniqueName: \"kubernetes.io/projected/d611de83-aa6f-401d-964c-6b87522cab23-kube-api-access-ss2tt\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.201624 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.201487    2524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-sys\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.201624 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.201550    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-podres\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.201624 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.201550    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-proc\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.201624 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.201588    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-sys\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.201896 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.201684    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d611de83-aa6f-401d-964c-6b87522cab23-lib-modules\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.209192 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.209125    2524 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss2tt\" (UniqueName: \"kubernetes.io/projected/d611de83-aa6f-401d-964c-6b87522cab23-kube-api-access-ss2tt\") pod \"perf-node-gather-daemonset-58fwj\" (UID: \"d611de83-aa6f-401d-964c-6b87522cab23\") " pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.269288 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.269255    2524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:58.399973 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:58.399948    2524 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"]
Jul 15 04:53:58.401956 ip-10-0-129-76 kubenswrapper[2524]: W0715 04:53:58.401911    2524 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd611de83_aa6f_401d_964c_6b87522cab23.slice/crio-41619f7c983fab859f63e01e062ec9c002ded7c8012ba8428ddf1d8f3a1aab8c WatchSource:0}: Error finding container 41619f7c983fab859f63e01e062ec9c002ded7c8012ba8428ddf1d8f3a1aab8c: Status 404 returned error can't find the container with id 41619f7c983fab859f63e01e062ec9c002ded7c8012ba8428ddf1d8f3a1aab8c
Jul 15 04:53:59.127561 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:59.127528    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj" event={"ID":"d611de83-aa6f-401d-964c-6b87522cab23","Type":"ContainerStarted","Data":"36398adef262666b5491a3f09fdc0246e5b7a87744d4558ccd23af6429f0db15"}
Jul 15 04:53:59.127987 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:59.127582    2524 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj" event={"ID":"d611de83-aa6f-401d-964c-6b87522cab23","Type":"ContainerStarted","Data":"41619f7c983fab859f63e01e062ec9c002ded7c8012ba8428ddf1d8f3a1aab8c"}
Jul 15 04:53:59.127987 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:59.127637    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:53:59.144333 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:59.144289    2524 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj" podStartSLOduration=2.144275913 podStartE2EDuration="2.144275913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:53:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:53:59.142523442 +0000 UTC m=+2868.453904705" watchObservedRunningTime="2026-07-15 04:53:59.144275913 +0000 UTC m=+2868.455657160"
Jul 15 04:53:59.456329 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:59.456252    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-9c55f886c-jz7wp_ede7204a-1a33-4f03-8cf3-01d5e7851923/console/0.log"
Jul 15 04:53:59.489955 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:53:59.489928    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-ht4kn_93c2aa95-043a-4825-b306-f64202da3c9c/download-server/0.log"
Jul 15 04:54:01.097874 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:01.097843    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h7bmp_a6b85d0d-2ca6-408b-b00d-60edb7fb41c7/dns/0.log"
Jul 15 04:54:01.117822 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:01.117797    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h7bmp_a6b85d0d-2ca6-408b-b00d-60edb7fb41c7/kube-rbac-proxy/0.log"
Jul 15 04:54:01.189482 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:01.189459    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kjdkn_3d7355a2-bc8b-435e-8c31-60297ecc4e35/dns-node-resolver/0.log"
Jul 15 04:54:01.761246 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:01.761220    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-696df47dcf-x5cn2_b925e347-51a5-425f-a836-6333d707ae6d/registry/0.log"
Jul 15 04:54:01.782309 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:01.782284    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9f68m_afb4f7f2-d8c9-4451-ae39-97507126f420/node-ca/0.log"
Jul 15 04:54:02.870631 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:02.870542    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-95pfs_b825790a-c79c-472d-9c56-ac3a248491bc/discovery/0.log"
Jul 15 04:54:03.032977 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:03.032947    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-7b9nq_adefd0a4-cd6b-4b91-9aa9-da45df2ca089/payload-pre-processing/0.log"
Jul 15 04:54:03.062941 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:03.062913    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-wrt6d_809162ae-0b69-4f2d-ba1a-7a97ec49f18c/payload-processing/0.log"
Jul 15 04:54:03.795212 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:03.795184    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-44srg_52b6b8d4-5eee-4b5d-96cb-befdb1d9b901/serve-healthcheck-canary/0.log"
Jul 15 04:54:04.394770 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:04.394742    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-slcc9_c9bef378-e519-4b5a-b871-e37044c90eae/kube-rbac-proxy/0.log"
Jul 15 04:54:04.413974 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:04.413952    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-slcc9_c9bef378-e519-4b5a-b871-e37044c90eae/exporter/0.log"
Jul 15 04:54:04.434128 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:04.434111    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-slcc9_c9bef378-e519-4b5a-b871-e37044c90eae/extractor/0.log"
Jul 15 04:54:05.141914 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:05.141884    2524 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lg2gg/perf-node-gather-daemonset-58fwj"
Jul 15 04:54:06.971833 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:06.971790    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67d859fc9c-kh8zv_aa2b64d7-833d-453c-bd68-0182c97e141d/manager/0.log"
Jul 15 04:54:07.241019 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:07.240947    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddd85b7f-dl9pg_863afdaf-88d5-467f-aaa4-c0a1af6bbfe8/manager/0.log"
Jul 15 04:54:07.266560 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:07.266530    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddd85b7f-dl9pg_863afdaf-88d5-467f-aaa4-c0a1af6bbfe8/copy-manifests/0.log"
Jul 15 04:54:08.401331 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:08.401303    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-95pfs_b825790a-c79c-472d-9c56-ac3a248491bc/discovery/0.log"
Jul 15 04:54:08.530637 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:08.530588    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-7b9nq_adefd0a4-cd6b-4b91-9aa9-da45df2ca089/payload-pre-processing/0.log"
Jul 15 04:54:08.551548 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:08.551508    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-wrt6d_809162ae-0b69-4f2d-ba1a-7a97ec49f18c/payload-processing/0.log"
Jul 15 04:54:08.559104 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:08.559070    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-c845c8db6-f69db_2823f9a0-14b6-41b6-aa5d-476f603c1267/manager/0.log"
Jul 15 04:54:08.583314 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:08.583292    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-4dsvx_6215b415-368a-4839-ba13-33014c84a73e/openshift-lws-operator/0.log"
Jul 15 04:54:09.951742 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:09.951711    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-95pfs_b825790a-c79c-472d-9c56-ac3a248491bc/discovery/0.log"
Jul 15 04:54:10.087873 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:10.087845    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-7b9nq_adefd0a4-cd6b-4b91-9aa9-da45df2ca089/payload-pre-processing/0.log"
Jul 15 04:54:10.111013 ip-10-0-129-76 kubenswrapper[2524]: I0715 04:54:10.110986    2524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-wrt6d_809162ae-0b69-4f2d-ba1a-7a97ec49f18c/payload-processing/0.log"